Grooveshark

[ZigZagJoe]

Thanks I have been looking at mplayer (after I tried wx's media controls (failed))... so ya I am probably going to use mplayer but I was thinking of passing the url for search result playing.

Only problem is you have to use post data to tell groovesharks' servers what to play. afaik, mplayer doesn't have a facility to pass post data.

Also, it looks like only the search results *sometimes* have genres.... I wrote a routine to use google to scrape genres, it's not perfect but it's the best i've seen so far. The code is in c#, but i can post it if you like.

[Zimmer]

I would love that, I know some C and C++ and can generally read code I can't program in (well get the gist etc.) so ya any code would be great! Thank you. :) (also is the source code for saver2 open (I am interested (curiosity)))

[ZigZagJoe]

I would love that, I know some C and C++ and can generally read code I can program in (well get the gist etc.) so ya any code would be great! Thank you. :) (also is the source code for saver2 open (I am interested (curiosity)))

Saver2/sproxy source currently isn't open but I do plan on releasing the savers (aka, the bits which sniff songs and gives the data to saver2 to do what you will with it). If i can find a good project host thing it's a distinct possibility though.

Code:

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.IO;

public class Genre
{
    public static string[] KnownGenres = { 
            "alternative", "other", "classical", "rock", 
            "hiphop", "hip hop", "r&b", "techno", "trance", 
            "dance", "punk", "pop", "metal", "rap", "jazz", 
            "ska", "electronic", "progressive", "country" ,
            "breakbeat", "funk", "world", "soul", "house"
        };

    public string Value = "";
    public string DebugResult = "";

    public override string ToString()
    {
        return Value;
    }
}

public class GoogleScraper
{ 

    private static string GetHTTP(string URL)
    {
        HttpWebResponse resp = null;
        HttpWebRequest req = null;
        StreamReader sr = null;

        try {
            req = (HttpWebRequest)HttpWebRequest.Create(URL);
            req.UserAgent = "Mozilla/4.0 (compatible)";
            req.ProtocolVersion = HttpVersion.Version10;
            req.Timeout = 10000;
            req.Proxy = new WebProxy("http://127.0.0.1:" + Proxy.CORE.Server.listenPort, false);

            resp = (HttpWebResponse)req.GetResponse();

            sr = new StreamReader(resp.GetResponseStream(), Encoding.ASCII);

            return sr.ReadToEnd();
        } finally {
            if (sr != null) 
                sr.Close();
            if (req != null)
                req.Abort();
            if (resp != null)
                resp.Close();
        }

    }

    public static Genre ScrapeGenre(string song)
    {
        return ScrapeGenre(song, " genre");
    }

    public static Genre ScrapeGenre(string song, string extra)
    {
        
        try
        {
            string googleResult = GetHTTP("http://www.google.com/search?q=" + Uri.EscapeUriString(song.Replace("&", "and") + extra) + "&num=30").ToLower();

            Dictionary<string, int> Counts = new Dictionary<string, int>();
            Genre g = new Genre();

            foreach (string genre in Genre.KnownGenres) {
                if (song.Contains(genre)) // it happens... unfortunatly
                    continue;

                if (!googleResult.Contains(genre))
                    continue;

                int c = 0;

                c += CountStringOccurrences(googleResult, " " + genre);
                c += CountStringOccurrences(googleResult, genre + " ");
                c += CountStringOccurrences(googleResult, " " + genre + " ");

                if (c == 0)
                    continue;

                Counts.Add(genre, c);

                g.DebugResult += genre + " w/ " + c + "\r\n";
            }

            if (Counts.Count == 0)
                throw new Exception("No results");

            List<string> resultGenres = new List<string>();
            g.DebugResult += "\r\n";

            while ((resultGenres.Count < (Counts.Count / 2)) || (resultGenres.Count == 0))
            {
                int maxVal = 0;
                string currMax = "";

                foreach (KeyValuePair<string, int> kvp in Counts)
                    if ((kvp.Value > 0) && (kvp.Value >= maxVal) && !resultGenres.Contains(kvp.Key))
                    { // find the highest which has not yet been added
                        currMax = kvp.Key;
                        maxVal = kvp.Value;
                    }

                if (currMax == "")
                    break; // just in case

                if ((resultGenres.Count != 0) && (Counts[currMax] < (Counts[resultGenres[resultGenres.Count - 1]] / 2)))
                    break; // don't add genres with few results when the big ones have been added already

                resultGenres.Add(currMax);
                g.DebugResult += "add " + currMax + "\r\n";
            }

            foreach (string s in resultGenres)
                g.Value += (char)((byte)s[0] - 32) + s.Substring(1) + " / ";

            g.Value = g.Value.Substring(0, g.Value.Length - 2);

            return g;
        }
        catch (Exception e)
        {
            throw new Exception("Failed to scrape genre", e);
        }
    }

    private static int CountStringOccurrences(string target, string str)
    {
        int c = 0;
        int i = 0;

        do
        {
            i = target.IndexOf(str, i);

            if (i == -1)
                break;

            i += str.Length;
            c++;
        } while (true);

        return c;
    }

}

Basically, google the song artist and track name, then count the number of occurrences of the recognized genre words. Then pick the top ones, up to (number of found genres/2) while the values being added are still at least half as big as the last one added. GetHTTP gets a page and returns it as a string. Certainly not pretty code, but it does work.

[gnounc]

love the program, Ive been looking for one in python specifically, so very nice.

So far in using it I've noticed:

*only 1 search works, after downloading and going back to search it errors

*the program doesnt create the download directory if it doesnt already exist.

*it asks you to agree to the eula everytime you start it

*it doesnt check to see if a song (or filename) exists before downloading

*obviously theres no option to stream song in player instead of downloading yet

*and it doesnt actually save the new filepath for next session

most of these have to do with persistance ala cfg file

Anywho, good work, cant wait to see how this evolves

[Zimmer]

Ya the new one should have some fixes but my main goal is to get streaming playback working (harder than it sounds :() ya I hope to have the next release soon.

[ZigZagJoe]

Ya the new one should have some fixes but my main goal is to get streaming playback working (harder than it sounds :() ya I hope to have the next release soon.

did a dirty hack to make your app output url and streamkey

doing so, "wget --post-data='streamKey=(KEY)' (URL) -O - -q | mplayer - -cache 320 -demuxer audio" worked fine. To control it, use mkfifo and mplayer slave mode. Only problem is that would limit it to *nix only. Until mplayer has an option to post data itself, you're going to have to break platform compatibility one way or another if you want to control mplayer.

[Zimmer]

mplayer can get piped data? Well that changes things :)

[ZigZagJoe]

for basic streaming support i added

os.system("wget --post-data='streamKey=" + data['streamKey'] + "' " + self.mp3URL +" -O - -q | mplayer - -cache 320 -demuxer audio")

right after

data['streamKey'] = self.songKeyfromID(self.songID)

in Grooveshark.py, and it streamed fine. Running on os x. though, the song download stuff didn't ofc becuase the stream key is consumed.

if you play around with piping to/from files you can probably have your cake and eat it too (stream it and save it). though, again, that would limit it to *nix only.

playing around,

wget (URL etc) -q -O "/home/user/wherever/file - artist.mp3" & sleep 3; mplayer "/home/user/wherever/file - artist.mp3" -cache 320 -demuxer audio -slave -quiet

causes it to be downloaded to a file AND played back by mplayer (after a 3 second delay, to allow for buffering). now, if you put commands into stdinput, you can control mplayer. "pause" and "volume [value] 1" are the most important ones, more on http://www.mplayerhq.hu/DOCS/tech/slave.txt

[Zimmer]

Well that will work because instead of using wget I can just do

while 1:

data = f.read(1024)

sys.stdout.write(data)

sys.stdout.flush()

if not data:

break

it will push the data to stdout in 1024 bit chunks as it gets it (then I just pipe it to mplayer)

[Zimmer]

one problem is how to control this... I wonder if I could do this in slave mode with pipes

[ZigZagJoe]

Well that will work because instead of using wget I can just do

while 1:

data = f.read(1024)

sys.stdout.write(data)

sys.stdout.flush()

if not data:

break

it will push the data to stdout in 1024 bit chunks as it gets it (then I just pipe it to mplayer)

Yeah, but you can't control mplayer without implementation-breaking pipes being involved if you are feeding the data via stdin. You could do something like (pseudocode, i don't know python)

system(wget URL -O file &)

sleep 3

process = popen(mplayer file -quiet -slave)

process.stdin.write("pause") and so on

and then throw commands at mplayer stdin while the process is sill running. Or, you could just use system() and let mplayer handle it (space play/pause, ctrl c quit)

for the pandora client, i end up using 2 threads plus mplayer (and the UI thread ofc). One reads data via http and stores the chunks received; the other feeds these blocks to mplayer via a named pipe. meanwhile, stdin and slave mode is used to control it.

to use pipes for playback

mkfifo pipe

wget url -O pipe

mplayer pipe -cache 320 -demuxer audio

[Zimmer]

hmm ya of course I could see what system they are on and use their native named pipes

[Zimmer]

Threads are evil! (what I got when I used them in python from people (due to problems debugging them

I like threads though

I think I got a solution

will be back with report on if it works

[ZigZagJoe]

hmm ya of course I could see what system they are on and use their native named pipes

the problem with this is quite frankly windows pipes are bitchy as hell and i rather doubt you could get them working sanely in python (c would have to be involved somewhere)

so, yeah, in conclusion it'd be best to have wget or something else output to a file (streaming) while mplayer reads from the file and you control mplayer via slave. It would be implementation neutral being the key thing.

[Zimmer]

it works!!!

Ok my solution

write to a file 1024 bytes of the file

then open it in mplayer

example (some mp3 I found (the audio I don't necessarily agree about (it is religion)))

I googled some mp3

http://www.ccel.org/ccel/anonymous/catacombs/mp3/

#Play
                file_ = open('C:/Kibble.mp3', 'wb')
                f = urllib2.urlopen('SOME URL')
                while 1:
                    data = f.read(1024)
                    file_.write(data)
                    if not data:
                        break

then in mplayer

mplayer.exe "C:/Kibble.mp3" (it works even if the file is still being download)

[Zimmer]

Updated...

url http://download774.mediafire.com/azdy55ssd...jnywno/dist.exe

Has streaming playback support - with controls

Creates download folder if it does not exist

Asks to overwrite download mp3s

Keeps settings over multiple app instances

Fixes bugs

----

multiple searches

[Zimmer]

Almost have a GUI done...

[gnounc]

The directory still didnt save properly.

Streaming works perfectly though, I'm going to test out playlists now, cant wait to see what you come up with for

a gui!

[Zimmer]

Ok gui is just about ready to go, playback works, download works... should be released later today.

[mtron]

Great ! Hopefully with source for us Linux guys ;)

Thanks

[Zimmer]

Of Course :) Heck without the cross platformness ZigZagJoe would have all the users (IMO his app is better (yes I am admitting my app is not the best))

[Zimmer]

Ok here is the source

http://pastebin.com/mbcc4bf

You need httplib2 installed (see first post), python (first post) and wxPython (wxPython.org/download)

[Zimmer]

GUI HERE FOR REAL!!!!!!!!!!!!!

Sorry about such a long time between releases.

For non windows users change the mplayer path in the settings tab.

Also you can change the default save location (defaults to Music directory in the apps directory)

http://download838.mediafire.com/l5m1wleem...distrobution.7z

Well it took me a LOT longer then I was hoping/thinking to make the gui but it is here...

You need wxPython for the gui app

- wxPython.org/download.php

Ideas for next releases:

Pandora support

Anything you guys can suggest (I am willing to add pretty much anything (that is not a promise though :-)))

Tips:

If you want faster releases, annoy and pester me :).

[Pichet]

Hi Zimmer,

I tried the *.py thing on a 'PureOS-Light' Linux LiveCD

with 'wxPython' and my evaluation didn't last too long:

missing stuff, whatever. Maybe an module would help!

[Zimmer]

pichet what stuff was missing? also why the hell do you want to run it on a live cd, this app writes files to the disk so a live cd WOULD NOT WORK. also on linux you need to change the mplayer executable from a windows exe to the linux version.