ZigZagJoe

Update. Slacker saver completed in full. Saves songs, w/ station name, big AART, etc. 337 lines in total. I'm quite amazed that the entire slacker player is written using AJAX. I need to implement a genre sniffer sometime, like pandorarip had, as it seems a lot of music services think genre metadata is optional. Currently working on manual song download. Shouldn't be too difficult. When that's done i'll release it as beta 2. Also, thanks the person that donated... you know who you are :)

oh, yeah, i've run into that too. i've ported the song fader("widget" to display last sniffed song) and pandora helper(pandora keyboard control for firefox and opera) to saver2, but it's not uploaded yet. (2nd page snipa)

that would be the definition of "weird bug" ... when does this happen? slow connection?

this is true, didn't think about caching the album art. though, i should note that pandora doesn't playu many songs from the same album (most is 5 ish i've found) album art reaping is already implemented - it defaults to 30 seconds (pandora tends to start downloading about 5% of song length before the next song comes, so it won't download it needlessly). if no AART in that time, the song is submitted and saver2 will download it. i do plan on adding a confirmation dialog when songs are still pending when sproxy is closing, a plugin can't stop the closing but it can delay it. Also, i wrote SProxy as a replacement host for pandorasaver, which used fiddler2. It doesn't have exactly the same featureset as fiddler, namely no built inspecting and stuff like that, but for plugin handling/web request handling it's superior IMO. (fiddler always caches the response before sending, leads to horrible memory use and also causes stuff to time out). It's not quite as complete insofar as HTTP protocol support goes either, It's currently at 1.0 while fiddler is at 1.1. I got kinda stuck implementing chunked transfer encoding and decided meh. SProxy is fine as a global proxy now, so that's good enough for me. (you can plug it in as a proxy for everything in firefox and nothing dies is what i mean) So yeah, if something needs changing, i can fix it. And that error in particular was related to copying a URL by double-clicking. I changed something a while ago and never fixed that. Woops. That line just tells me where in the source it died on my computer, it has nothing to do with the executable. in this case, it was a bit to cut off the front of the text (get rid of GET and flags). Except i removed flags, so it was looking for something that was not there. AJAX: Yeah, true, i could have a thing which checks for all marked cells, but the problem is establishing ownership - SProxy won't know what "songsripped" is nor who does know it; however there is no nice way to signify that in the source. Also, each plugin that wanted to make use of that would have to have a response set for each variable. So it'd get somewhat tedious. I'll probably just go for auto-refresh as a hidden setting. (edit under advanced settings). Anyways, updated with the song aart caching and that sproxy bugfix. I contemplated having it respond with saved data in pandora, but it's still made the request already so might as well ready the 8kb or so it's sent. Also, album art is saved in png rather than the jpg that pandora needs, so that would have to be converted. But, if saver2 has to download the aart and it's already present it will used the saved one (if that feature is enabled) Random statistics: Saver2 contains 2657 lines of source Pandora contains 850 lines of source SProxy contains 5005 lines of source Deezer contains 276 lines of source DatLib contains 266 lines of source

could do yeah if i were really clever i'd use AJAX but that'd be something of a pain to do as each plugin would need to maintain their own ajax parts by selective song downloading, i meant having all songs that were sniffed (and not blacklisted, when that is implemented) being stored instead of immediatly ripped and at the user's discretion they are saved or discarded. conversly using the blacklist as a whitelist in this mode means songs that match blacklist patterns are automaticly saved (add a favorite artist or something and go wild) however, yes, i plan to move duplicate detection to the sniffed stage and if a song has duplicate discard the rest of the server's response and use a 5 second blank file. Edit: Implemented shunning and uploaded same link. Songs that already exist and can't be upgraded (if enabled) are not downloaded if in turbo mode, and the next song is played in 5 seconds. (no less to be nice to pandora servers)

Eh. I'd rather use the desktop OS i'm using rather than startup a CD _just_ for burning. Also, if you had to do any different config (say you need to use Tor) it'd be the same on a livecd as it would normal OS. Still gotta set it up/configure it etc. Otherwise, it's extract a zip file and off you go. Seems easier. Easier still if you use pre-configured firefox portable. If you really want a livecd now, look into bartpe, stick .net 2.0 or so on it and just as good. It's not linux, but at least it'd work.

re-download it if you are using turbo - there was a flaw relating to random name generation and turbo mode which caused songs to not be downloaded (due to using the same temp name for all the songs sniffed at once - yay psuedorandom random generator). I've also added an entry on the status page for it, you now get to know known stations count, how many songs are known, avg download speed, turbo mode status, adn some other bits. Most importantly, it shows pending songs and their statuses such as waiting for aart Daft Punk - "Aerodynamic" on 'Discovery' (ID S87105) downloading now Marilyn Manson - "Sweet Dreams (Are Made Of This)" on 'Smells Like Children' (ID S262798) waiting to download Future Prophecies - "Concrete Combat" on 'Moving Shadow 05.2 - Live Mix By Calyx' (ID S370446) More importantly, I've added a throttle function to sproxy. go into the settings, and enter the desired download rate in KB/s (up to your connections max ofc). Do note that throttling assumes an equal bandwidth distribution for all active requests, so the end download speed available to one connection is throttleTarget/numActiveConnections. So with 2 active connections, and a 300kb/s target, each gets about 150kb/s. Why does this matter? it will cause turbo to mess up for 3 songs or so if throttling is active and you are making other requests via sproxy besides pandora, as it thinks your download speed is much higher. (it uses average of the last 3 songs to compute turbo wait time) I've made motions towards getting the selective downloading implemented (status page RPC and form, + trayicon). Also, instead of integrating the DatFile, which is used as a "simple" resource storage mechanism, more elegant than using a zip and some outside library, i've made it into a lib. To use it, reference the dll. Methods are relatively simple-> static bool DatFile.Verify(string filename): verify the md5 hash of the payload static void DatFile.Create(IList files, string destfilename, bool useCompression): create a new dat file non-static methods as follows Constructor(): create a new datfile object. does nothing important bool Compressed: true if using gzip compression, false otherwise int Count: number of entries in dat file read byte[] GetData(DatEntry entry): decompressing an entry if neccisary, it returns a copy of the data for it byte[] GetData(string entry): same as above, except uses an entry name instead of entry struct void Read(string filename): read a datfile from disk. verifies md5 before reading this[int index]: returns datentry for index this[string key]: returns datentry for key (entry name) compression is gzip. in the case of the turbo data, which was 14 mp3 and m4a files (blank), ~500kb uncompressed, it reduced it to 14kb. entries are uncompressed on demand rather than as a whole for sake of performance. in addition, when an entry is uncompressed it remains uncompressed for performance reasons as well. zipping the lot gets it to about 8kb, as the gzip entries don't share dictonaries. Nor is there a way to have c# gzip stream support multiple files in one gzip entry - it's all or nothing. simple GUI to make dats: http://zzj.itf-inc.com/downloads/DatMaker.exe name of entrys is the filename, minus any slashes (thus C:\Documents and Settings\SProxy.exe becomes SProxy.exe"). Case sensitive. plugin "interface" is relatively solid now, i'll post about it later. (interface being how saver2 talks to its plugins, not a true c# interface)

Okay, i've updated the link above with a new (still beta) version. This is a mostly pandora update: Pandora Greatly improved turbo mode (!!) Configuration dialog Some bugfixes Saver2 Fixed newmusic not having ext Plugin infrastructure additions Sproxy Various internal bugfixes Turbo changes are basically it's much more kind to pandora's servers now, you only need to run one pandora at a time in turbo (moreover more than one will prevent it from working properly) + it should not fail due to a song taking too long to download. Other changes: Low bandwidth mode is obsolete and has been removed. All songs will now start playing immediately, and so on. (in effect, it's always in low bandwidth mode) Known bugs: if you exit pandora while a song download is in progress, but pandora has not changed to the new song, it will never be saved. (song never displayed -> pandora never displays aart -> song never ripped as aart is the trigger when present) Notes about how the changed turbo mode works: Instead of sproxy download all of a file and then let the plugins have a whack at it before sending it off to pandora (or whomever requested it), it asks the plugins first if it should handle it specially. it is up to the plugins to mark no streaming (so that something can be inserted into the source, like the pandora main page auto refresh), or to cache the song so that plugins can look at it at all (by default, files are not cached to save RAM, as sproxy does not have an inspection interface there is no point). It's also possible for a plugin to respond to a request immediately and either have sproxy not download what the server is sending or download it and send to plugins only. (how turbo mode is done) %source% tag added for saver2 file naming/dir organization stuff. a note for people making their own pandora saving dealie: this will save ALL xmlrpcs to the temp(in sproxy dir) folder! End up with stuff like PANDORA_ADDFEEDBACK_i6ul5Gb00Zk7bjl0.XML, PANDORA_GETFRAGMENT_YDaWDZ404gSJ81mu.XML, PANDORA_GETSTATIONS_p5UnMb9P3x5X1RFa.XML, etc. Also, i noted the introduction of a canListen method - perhaps this is the way they disable listening over 40 hours or something? I'll dissassemble the flash sometime and check. Could always rewrite it to be 1. I'm looking into making an Imeem saver as i like it when i'm just lookin for a certain song... its metadata is actually very similar to deezer's (very, very similar). But the thing i am stuck on at the moment is that i need to determine how to relate a file to metadata. Grooveshark, slacker are on the to-do list but grooveshark requires me to check out the metadata again. slacker is ready to implement once i confirm nothing has changed.

Yeah, i mean to try it sometime. Re-encoding wouldn't work without wine (Or maybe replaced with linux binaries?) but the rest is fairly standard C# code, nothing too exotic. (that i can recall)

just curious, why do you need a livecd so much?

One of the first. Another note - I'm working on an improved turbo mode (and general plugin handling). This method prevents pandora from getting fidgety as it responds with a blank file immediately, of either 5, 10, 15, 20, 30, 60 second length, picked based on how long it thinks it will take to finish downloading the current file. Also, i should note that the old pandorasaver is incompatible with the new sproxy, and same deal for saver2 - i've had to make changes to the plugin interface so it will fail if you try to load the new with the old or vice versa.

Time for the first beta. Creating a new thread for Saver2: http://hak5.org/forums/index.php?showtopic=13598&

Saver2 - http://ridetheclown.com/wp/saver2/ DO NOT REPLY TO THIS THREAD - I WILL NOT SEE IT! INSTEAD, GET ON IRC BY CLICKING THE "LIVE HELP" BUTTON ON SPROXY, by launching IRCHelp.exe in your install folder, OR by joining #saver2 ON irc.coldfront.net ! You can also get on irc by going to this web page.

To-Do updated. At the moment, blacklisting is going to be indefinitely delayed unless specifically requested. Simply put, I don't like the implementation of it but I can't spare the time to sort it in a way that will satisfy me. I'll probably get it sorted sometime, but it will not be in the initial release. I'm not sure if i'm going to do tray icon immediately or not. Depends if i can find some sort of icon. lol I'm wondering, would people find some sort of notification a song is pending ripping and/or was ripped automatically? Say, a tray popup? Probably going to open-source the Savers at very minimum (the plugins which talk to CoreSaver, the main bit) cos people might be able to benefit from implementation stuff within. Coresaver on the other hand just contains the glue that handles the other bits - not nearly as important. SProxy will stay closed source for the conceivable future, though i plan to expose the bits of the plugin interface needed to implement plugins. Turbo now ported and working successfully with Pandora One. Stuff that needs to be re-evaluated: Upgrade mode. If not the same type, then references get broken when the old file is replaced (i have 12k or so aacplus pandora songs, and now that i have pandora one, i'd like to convert those to the higher quality mp3. however, if it just blindly replaces songs, then any playlists that refer to that file will be broken. also, pandora's playlist creation would then proceed to insert the song a second time...

An update on things. Just had Saver2 rip its first song earlier. Still got a bit to do, though. Misc. Now portable-friendly, as it does not use system temporary folder at all. To Do: Implement encoderworker Implement song fader as a seperate class Ripping selection dialog (pick which songs to save, port from ps) status page song downloading Pandora: tuner Pandora: tuner web rpc Pandora: configuration Logo Tray icon Done: Pandora: playlist saving Pandora: turbo Pandora song sniffing FLV de-encapsulation Song processing Add %exinf% tag Config form controls Plugin template Web Download RPC All AArt Handling Pandora: Save XMLRPCs Configuration form Song Handling On Hold: Port pandorasaver blacklist class Whitelisting Find service to auto-complete with Pandora: javascript songplayed notification (only would help in a few cases... maybe later) Other: Log viewer form for SProxy (from pandorasaver) Revised settings class.

Yes, those are the mp4s. Which are somewhat useless (to me) if they have no tags (which is why you have to sniff the metadata). There is only one token per unique song. Sounds like downloadhelper is listening em twice.

Sorry for the lack of response, I'm a bit busy with RL at the moment. But I am working on it. (Saver 2 to be exact). As part of saver2 will be a template class which can be inherited (for someone to make their own ripping plugin). It contains the logic which allows saver2 to recognize a plugin loaded as sproxy as a saver-plugin (which may submit music to it) and allow for saver to query the plugin for information and pass on events. The general concept is fairly simple, the plugin will override a method called (subject to change) SniffMusic which is called with a session object, the plugin figures out if it song data or what not if it is creates a SongInfo object, saves the data to a temp folder, puts in at minimum the artist and title, and then passes it to coresaver. The plugin will need to keep track of its own metadata and attach it to song data, but from there core saver handles it (blacklisting, approval by user, re-encoding, etc etc etc). Nice and modular. It will even complete the song data via gracenote or cddb or something and fill in missing fields. Song data format does not matter, as long as mplayer can play it it can be converted. Otherwise, it will be saved in the native format (flv containing mp3 will be extracted, however.) Additional functionality currently allowed for is a custom status report, custom unload, onsongripped (called when a song sniffed by the plugin is approved and processed), setup menu to allow custom menu items, support for a custom menu (DoConfig), and web RPCs. A log function is also available (void Log (string stuff)) which puts stuff into sproxy's log prepended with the plugin which called it. In short, it'll be a lot easier (compared to how it is now) to write plugins which rip stuff. Of course, I'm going to update pandora, grooveshark, and deezer for this, and likely implement slacker.

This is the "official" thread, yeah... no real site at the moment though. I had kinda hoped to finish renovating my site before this went so far, ol.

Ach, to all people who downloaded the SProxy1003dPS1008e.zip file and are getting an error on start: Delete Sproxy.conf from the directory you extracted to and try again. Minor booboo.

What happens?

Have you tried the new one? Anyways, I've started work on a completely new codebase, which will definitely support pandora one. Stuff of note: Multicore encoding (not for a single song, encoder limitation, but if there is more than one song pending encoding on a multiple core machine it will run up to nrCores worth of encoder processes) Support new music services without re-writing an entire new plugin- use a unified blacklist/song queue/encoding queue/etc. Settings too "Upgrade" feature, where if it finds that a song already exists, but the song pending ripping is higher quality, it will replace the old one (probably for mp3s only, as it would be a pain to reencode into aac/wma/otherwise and it would break existing playlists if it did) I won't strangle myself looking at the code anymore Most pandorasaver features will be ported over (given time). Of course, it's still in very basic alpha stages. But it's coming. I plan to release the source to the current one when i get this to a stage ready for release (supports most of the core pandora features). A warning, it's ugly. Main source file is 74kb in size. Assuming my computer doesn't off itself. Had the main data HDD (with all my pandora music :( , anime, etc) die and possibly another. Luckily I don't have code on that hdd.

// script which IE or firefox can use to decide how to route a certain URL // this can also be used to avoid pandora's US-only restriction function FindProxyForURL(url, host) { if (shExpMatch(host, "*pandora.com*")) // url contains pandora.com return "PROXY 127.0.0.1:8888"; return "DIRECT"; // no proxy } In the first post :S Also, IE seems to be a bit weird evaluating this. (I've not tried it in a while though)

Would help, yes. I've made a version which I think SHOULD work with pandora one, but no guarantees: http://zzj.itf-inc.com/downloads/SProxy1003dPS1008e.zip Grooveshark and Deezer savers are now in a separate zip at http://zzj.itf-inc.com/downloads/Groovesha...ezerSaver01.zip , because they increase the size I have to upload by about 4mb (and I've not updated them yet, so it's a waste of time). Extract the zip into the folder where SProxy.exe is to install them. Changes: Cached DNS resolver improved. GS and DS split into seperate zip. Option to disable automatically setting IE settings to use sproxy. Various core improvements. Quirks: Re-encoding will NOT work with pandoraOne and is disabled. PS uses a specific aac decoder, not a generic one like mplayer. It is possible to have it use Mplayer but until it is specificly requested I'd rather not as that increases download size by 4mb or so (unless I try to compile mplayer with just audio codecs or something; but getting all the libs together is like pulling teeth). I do not know if turbo mode will work with pandoraOne or not. Behavior is undefined; I'd have to look at the disassembly to try to figure it out.

Possible. I think i may have seen a length attribute in the metadata, but there is only one way to find out how it will react. I usually just click play on a song and when i see it start downloading in sproxy hit play on another, and so on. All of them get downloaded (except if ID3 fails, which i still need to sort)

not an option in the current version but might add it in the "service-pack" which i hope to release in a few days. just under the hood fixes in it, nothing major really. Not automatically setting IE options is the only real change that you can see