liquidcool Posted October 22, 2008 Share Posted October 22, 2008 Fiddler is an http proxy for debugging http traffic - even SSL traffic. It supports plugins written in .net. You can download it from www.fiddler2.com I have written a plugin for fiddler2 that does the following: * watches traffic to pandora for xml documents describing a playlist and remembers audio url and id3 info * watches for requests to a known audio url and saves the recorded response as .mp4 * runs faad to convert the mp4 to wav * runs lame to convert the wav to mp3 * inserts an id3 tag in the mp3 with artist, album, title, and covert art image * deletes the 2 temp files (mp4 and wav) Also it: * watches for requests to pandora.com/ and injects a meta-refresh right into the source of the page to force it to reload periodically so it won't stop playing. To run, copy the dll and 2 audio conversion exe's into the fiddler scripts directory and start fiddler, then start ANY browser (IE is automatically hooked by fiddler) and my fiddler plugin will start capturing music to the folder of your choice. This method for timeshifting doesn't rely of flash or browser or caching or any other miscellaneous technology. It watches everything going by and saves the music. Apparently I can't upload zip files here and I'm not going to host it myself, but if you want the source code, drop me a pm. Quote Link to comment Share on other sites More sharing options...
Birra Posted October 22, 2008 Share Posted October 22, 2008 Fiddler is an http proxy for debugging http traffic - even SSL traffic. It supports plugins written in .net. You can download it from www.fiddler2.com I have written a plugin for fiddler2 that does the following: * watches traffic to pandora for xml documents describing a playlist and remembers audio url and id3 info * watches for requests to a known audio url and saves the recorded response as .mp4 * runs faad to convert the mp4 to wav * runs lame to convert the wav to mp3 * inserts an id3 tag in the mp3 with artist, album, title, and covert art image * deletes the 2 temp files (mp4 and wav) Also it: * watches for requests to pandora.com/ and injects a meta-refresh right into the source of the page to force it to reload periodically so it won't stop playing. To run, copy the dll and 2 audio conversion exe's into the fiddler scripts directory and start fiddler, then start ANY browser (IE is automatically hooked by fiddler) and my fiddler plugin will start capturing music to the folder of your choice. This method for timeshifting doesn't rely of flash or browser or caching or any other miscellaneous technology. It watches everything going by and saves the music. Apparently I can't upload zip files here and I'm not going to host it myself, but if you want the source code, drop me a pm. Am I misreading it, or did you just forget to put the url to your plugin? Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 22, 2008 Author Share Posted October 22, 2008 Am I misreading it, or did you just forget to put the url to your plugin? source and binaries: http://wikisend.com/download/511348/PandoraSaver.zip let me know how it goes and I am not responsible for misuse. Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 22, 2008 Share Posted October 22, 2008 This is very nice, but one question, which dll/exe files? I am assuming the ones in "PandoraSaver\bin\Debug" folder? Also, would it be all dll/exe files, or just certain ones? Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 23, 2008 Share Posted October 23, 2008 Ok, I figured it out, and am now posting instructions on how to do it...... First off.... 1: Download/install Fiddler2 2: Download PandoraSaver.zip Setup.... Create a folder on your desktop called "Pandora Saver" and extract PandoraSaver.zip into it. Now, open up that folder on your desktop, then go into the folder "PandoraSaver", then into the folder "bin", then into the folder "Debug". you should see the following files: faad.exe lame.exe PandoraSaver.dll ect..... Copy ALL of these files into fiddlers script directory ( Normally found at "C:\Program Files\Fiddler2\Scripts" ). Usage...... Now, follow the following steps carefully or else it WONT WORK. Internet Explorer is recommended for this (firefox does work, but for simplicity reasons, lets just stick with I.E. for now). If you have I.E. open, close it. Start up Fiddler. If you installed the scripts properly, you should see a button on the toolbar called "PandoraSaver", click it. A dropdown box will appear. Here you can configure it as you see fit, but remember, when you close Fiddler, these options are not saved. Once you have it configured, click the button and click on "Enabled". This should put a check-mark next to it, which means the script is now running. Now fire up I.E. and go to pandora website. Fiddler will now capture the songs and put it in the folder you specified. Pros: I.E. does not need focus in order for this to work, a BIG PLUS in my book. Saves the song/album and album art, a nice feature. Cons: Does not save the songs in seperate folders (not a big deal, just would be nice for organization freaks such as myself ;) ). MP3 bitrate is 128kbps (not bad quality, but myself would prefer higher quality) BONUS: As a test, I tried this with the pandora gadget for windows vista (to find it on the windows gadget website, just do a search for "Pandora"), and guess what, IT WORKS! To use the gadget with this, just follow the same steps above but: Be sure the gadget is not playing any songs, start fiddler and start the script, then press play on the pandora gadget. This thing is very, very nice indeed! Final verdict: This is the best one yet. You can be doing whatever (playing a PC game, working on a spreadsheet, ect) and this thing will keep churning along. Tested with Vista Ultimate 32bit and works great. Questions to the author of this: Does this embed the album art into the mp3, or does it pull it from an outside url? Can the bitrate be increased, or is this the bitrate that pandora streams at? Would it be possible to store the songs in seperate folders like "Artist/Album/Artist-Song.mp3"? Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 23, 2008 Author Share Posted October 23, 2008 Questions to the author of this: Does this embed the album art into the mp3, or does it pull it from an outside url? Can the bitrate be increased, or is this the bitrate that pandora streams at? Would it be possible to store the songs in seperate folders like "Artist/Album/Artist-Song.mp3"? Answers: * album art comes from pandora, but since pandora includes the amazon isbn, it wouldn't be hard to fetch the larger image from amazon * The bitrate could be specified in the lame encoder step - to be honest i'm not sure what the optimum setting would be - you could encode the wavs at 300kbs, but at some point you're encoding far higher than pandora serves them. i guess it would be ideal to give you a setting to specify command line switches for lame.exe * of course - you could do anything with this. The source is included so someone else could pick this up and run with it. I've only invested a couple hours in this and don't have the time to maintain it. I was just interested in a proof of concept to demonstrate a different method that doesn't rely on specific browser version or plugins since it operates at the network level. Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 23, 2008 Author Share Posted October 23, 2008 I would also add, it wouldn't be hard to add support for deezer.com and other online radio sites. Quote Link to comment Share on other sites More sharing options...
gen0cide Posted October 23, 2008 Share Posted October 23, 2008 This really has me thinking. The xml data points to the related and possible upcoming songs, with title, album, genre, 130 x 130 album art. So all this operates at the Layer 7 model. I think, Pandora uses the blowfish algorithm to tell the laszlo servers what xml file to throw at it, which the xml file includes the audio source and song info and other songs that could play after it. I was curious so I decompiled the Pandora's swf, at https://www.pandora.com/radio/tuner_8_6_0_0_pandora.swf , and there's 50,000 lines of actionsript code in the MainMovie . It is hard to understand, it has alot of laszlo server specfic bs in it. <_< Now, we have access to the xml files when pandora hands it to us, which include the song info. We could make our own media box, like the squeezebox, that displays current song info, which could be outputted to a lcd. An old thin-client would be the best for this. It would be nice if everyone had a stand-alone application, that monitored the layer 7 traffic, and parsed only the Pandora xml files, while pandora swf was running, like in a self contained window, like all those wrappers that are out there. Making something like this with the QT toolkit would make this alot easier I think, and would make it eaiser porting it to multiple OS's. These are just my thoughts ;) liquidcool , your code looks very good and staight foward :) I think it will be you that launches an ultimate Pandora project foward, its a big step foward. We just need to figure out how the pandora talks to the laszlo servers while using their blowfish encryption, then we could develop our own standalone pandora app, that woudnt require flash, as flash is a resource hog. All in all, Pandora is a company, built upon the capitalistic model, and the only reason they are still providing their services for free are the ads. If it wasn't for those advertisements, Pandora would not be here today. So I say we should click on their ads for every couple songs you listen too, I think that should be a fair agreement between us and them. Here is the decompiled form of the Pandora tuner, compressed with Rar, if anyone wants to look at it: http://rapidshare.com/files/156914380/tune...andora.rar.html Quote Link to comment Share on other sites More sharing options...
Birra Posted October 23, 2008 Share Posted October 23, 2008 This is not working for me, I think because I already am behind a proxy (HotSpotShield). Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 23, 2008 Share Posted October 23, 2008 I know next to jack about programing ( I have played around with autoit and made "hello world" in C++, but thats it). My question is what did you use to make this so I can inspect and play around with it a lil (was it done in C++, java, fiddler addon, ect)? EDIT: If I would have opened up my eyes earlier I would have seen that you already stated that it is done in .net..... Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 24, 2008 Author Share Posted October 24, 2008 I know next to jack about programing ( I have played around with autoit and made "hello world" in C++, but thats it). My question is what did you use to make this so I can inspect and play around with it a lil (was it done in C++, java, fiddler addon, ect)? EDIT: If I would have opened up my eyes earlier I would have seen that you already stated that it is done in .net..... Well the nice thing is that building a proxy for all http traffic would be a huge pain in the ass. Fiddler makes this easy as its plug in architecture allows you define a function that is called whenever an http response finishes. So now you have the url and all the response data and you can do whatever you want with this and you don't have to worry about the details. Fiddler exposes methods like SaveResponse(filename) and uirContains(string), etc. If you look at the code, it simply says, is this a pandora xml file? if yes, remember the audio urls and data. Is this an audio url that we saw earlier? If yes, save it and tag it. So it will work with any pandora client running over http including firefox, ie and the pandora widget and task tray app. Incidentally I went to deezer.com with fiddler running and it's basically the same thing. They don't use xml, but before the music plays, there's a file transmitted with all the artist/album/song details and a songID.. then later your browser issues a request for that songID. The song itself is an FLV format file, but it would be easy to add another exe that converts FLV to wav and then does the lame.exe step and tagging as well. If someone went all out, you could build this plugin to work against any radio site out there. Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 24, 2008 Share Posted October 24, 2008 Just by manually looking at it through fiddler, I understand how this works, but what is confusing me is how it matches the audio stream with the artist/album/art info in the xml file. Is there something in the header in the audio stream that I am not seeing that matches up with something in the xml file? I am about to install VS2008 express editions (the free ones from MS site) so I can look at your source so I can get a better understanding on how it matches this up, unless you can "drop a hint" here on it, lol. All in all, this is very, very good work, ty for your proof of concept. I also beleave that what you have done here will start a whole new trend on how people will move forward with capturing streaming audio. Quote Link to comment Share on other sites More sharing options...
AnonymousCoward Posted October 24, 2008 Share Posted October 24, 2008 The xml file has the url of each audio track. If you capture for the reponse to the HTTP GET request of that url you will have the mp4 audio file. My (as yet unpublished) capture utility works in a very similar manner as liquidcools but I use a Wireshark library to capture packets. This is my first I've seen Fiddler and I'm liking it. I may rework my program to integrate with fiddler instead as I like it's session based capturing better than having to reconstruct the sessions myself. Thanks liquidcool! As a side note, I've, in the past, steered clear of talking specifics about what I've learned about how the exchange between the browser and pandora happens lest they take notice and obfusticate it beyond use. "But that's just my opinion, I could be wrong." Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 24, 2008 Share Posted October 24, 2008 I did notice that, but unless I am looking in the wrong spots in the xml, the URL's dont exactly match between what is in the xml, and what is actually streaming....I will look into it more to so I can better understand it......god I wished I knew C# BTW, I do agree with your point on "openly discussing" on how this all works. Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 25, 2008 Author Share Posted October 25, 2008 I did notice that, but unless I am looking in the wrong spots in the xml, the URL's dont exactly match between what is in the xml, and what is actually streaming....I will look into it more to so I can better understand it......god I wished I knew C# BTW, I do agree with your point on "openly discussing" on how this all works. You're right, they don't exactly match - that's why I'm comparing only the first 350 characters. Those match exactly and the extra chars are some kind of salt code. But if the first 350 are the same, it doesn't matter what's on the end. Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 25, 2008 Share Posted October 25, 2008 Ahh, I was beginning to suspect that, but I was not sure.....basicly, their way of tricking a basic noob such as I, lol. BUT, I wounder if this is the case, or is it a limitation of fiddler? I do beleave my first statement is more correct though... I am in the process of taking a "crash course" in C# in my spare time so I can learn more about this. Quote Link to comment Share on other sites More sharing options...
lerxst Posted October 25, 2008 Share Posted October 25, 2008 Can anyone post another link to the source/binaries? I don't want to download "Alan.Dec6 address.list" from the link given above on wikisend. Thanks! Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 26, 2008 Share Posted October 26, 2008 Here is the origional zip file that I re-uploaded to wikisend. Also, I upped it to 90 days that wiki will hang on to it. LINK: http://wikisend.com/download/526724/PandoraSaver.zip Quote Link to comment Share on other sites More sharing options...
lerxst Posted October 27, 2008 Share Posted October 27, 2008 This is great -- good job! I do have one little issue I haven't quite figured out yet (it could be user error). The logs are posted below, but it appears that what is happening is that the first 2 songs are captured and then all subsequent songs return a message about not being able to find the WAV to convert to MP3. I changed the log message and recompiled to make sure the right faad.exe was being called. Any thoughts before I start hacking away at your code? Thanks, Lerxst 10/26/2008 8:43:55 PM Saving: http://audio-sjl-1.pandora.com/access/8304...zC3cGJHZCTRI%3D 10/26/2008 8:43:55 PM Length: 3433951 10/26/2008 8:43:55 PM Running C:\Program Files\Fiddler2\Scripts\faad.exe "D:\Shared\Music\dnld\Pandora\Rihanna - Break It Off.mp4" -o "D:\Shared\Music\dnld\Pandora\Rihanna - Break It Off.wav" 10/26/2008 8:43:55 PM Didn't find the wav file 10/26/2008 8:43:55 PM External processes done Quote Link to comment Share on other sites More sharing options...
lerxst Posted October 27, 2008 Share Posted October 27, 2008 Just a quick update on my error above. I pointed to a batch file instead of faad.exe and added a pause to that so I could see the output from faad.exe and I'm getting "Error initializing decoder library" when it's trying to decode the downloaded MP4. The downloaded file is in fact not a valid MP4 because it won't play in QuickTime. I can send you a copy of the downloaded file if you'd like. Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 27, 2008 Author Share Posted October 27, 2008 Just a quick update on my error above. I pointed to a batch file instead of faad.exe and added a pause to that so I could see the output from faad.exe and I'm getting "Error initializing decoder library" when it's trying to decode the downloaded MP4. The downloaded file is in fact not a valid MP4 because it won't play in QuickTime. I can send you a copy of the downloaded file if you'd like. this command from the log: C:\Program Files\Fiddler2\Scripts\faad.exe "D:\Shared\Music\dnld\Pandora\Rihanna - Break It Off.mp4" -o "D:\Shared\Music\dnld\Pandora\Rihanna - Break It Off.wav" converts the mp4 from pandora into a WAV that we can convert to an mp3. I'm guessing if you can't play that file, then it's either not a valid mp4 or you don't have the right codecs installed. Hmm, seems very odd that the first 2 songs work and subsequent ones don't. I'm sorry, I don't have an answer for you. Perhaps some of the other developers will run across this and identify the problem. Quote Link to comment Share on other sites More sharing options...
MoparMan Posted October 28, 2008 Share Posted October 28, 2008 This made me curious, so I deleted out my log file (so it would be fresh), ran the pandora gadget, and payed close attention to what song was playing and what the log showed.....and I found something.........odd From watching the gadget, everything seemed fine, but from the log, it looked like pandora was prepared to play a certain song, then changed there mind shortly after and switched to a different one. confused? let me make a mockup log so you would understand (Note, sections in blue are correct, the red section is what caught my eye)..... ect.ect.ect...... Found mp3 file (URL).... saving....... Length: 1805812... running (ect.ect.ect)... running (ect.ect.ect).... external process done.... ect.ect.ect...... found mp3 file (URL).... saving...... Length: 205 running (ect.ect.ect)........ Didn't find the wav file External processes done ect.ect.ect...... Found mp3 file (URL).... saving....... Length: 1302598... running (ect.ect.ect)... running (ect.ect.ect).... external process done.... ect.ect.ect...... Notice the song length in the red section? From watching pandora, everything seemed fine, but from the log, one song played, then it looked like pandora started prefetching the next song to play, but right after prefetch started, panora decided not to play that one, dropped that song, and started prefetching the next song but from watching the pandora gadget, everything seemed fine. I also would like to point out that this is the first time I noticed this, and I have been using the pandora gadget since this came out (and yes, I do watch the log file from time to time just to be sure that this is working as it should). I also noticed that this "bad stream" is completely random from what it seems. Warning, personal assumption/guess ahead....... I believe that what is going on is that this is pandora's attempt to fill our song library with bad, dead songs, but due to the encoder's error checking, along with the programing of deleting *.mp4 and *.wav files, all traces of said song are removed. OR.... this is mearly a network issue/pandora server issue that caused an error in streaming said song, forcing it to be dropped/skipped and move onto the next on. Quote Link to comment Share on other sites More sharing options...
liquidcool Posted October 28, 2008 Author Share Posted October 28, 2008 Well I think at least we know why the encoding failed... length is the number of bytes in the file and I doubt those 205 bytes it saved were anything of substance. Quote Link to comment Share on other sites More sharing options...
adctz Posted November 15, 2008 Share Posted November 15, 2008 Does anyone have an alternative link for this file? Does it still work? Quote Link to comment Share on other sites More sharing options...
MoparMan Posted November 16, 2008 Share Posted November 16, 2008 The link I posted still works... http://wikisend.com/download/526724/PandoraSaver.zip and, yes, this app still works as long as you live in the U.S. (due to pandora's licensing ) Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.