if its too loud, turn it down

Monday, June 29, 2009

Use perl and applescript to keep playlists synced with iPhone/iPod

I am always adding to and editing iTunes playlists, particularly the more oft-listened ones like my "punk", "ambient", and "programs & podcasts" playlists. Since I listen to alot of stuff on my iPhone, I naturally want these playlists always available and up-to-date on my device. I can't tell you how many times I plugged in my iPhone to my car stereo and found, to my disappointment, that I forgot to drag-and-drop the most recent version of a playlist to the device.

Why not just set the iPhone to automatically update? Because I want the finite control that manually managing affords me. (UPDATE: to all you haters out there wondering why I don't just use auto-sync...because it "locks" the iPod so you can't manually manage any playlists. I want to manually manage most things and just have selected playlists update automatically. If you don't like this script don't use it!)

So I created a script that updates a specified list of playlists to all attached iPods/iPhones. It can run on a schedule (with cron), so I never have to worry that my iPhone won't have the most recent versions of my playlists!

Features
  • Adds specified playlists to all iPods attached to your mac
  • Will skip the playlist if there's not enough space on an attached device
  • Retains the order in which the tracks appear on the library playlist
  • Outputs useful debug info
Usage

First, the script requires the iPod/iPhone be set to "Manually manage music and videos". I named the script "updateipod.pl" and placed in my user's "bin" directory. It's recommended you specify the playlists that you want to sync in the @playlists array at the top of the script:
my @playlists = (

 "jamz",
 "0pods",
 "punk best",
 "stoner best"
 
 );
After chmod-ing the script to 755 just invoke it as you would any script:
/Users/user/bin/updateipod.pl
Optionally, you can specify the names of the playlists as arguments on the command line. Useful for one-off updating. Note that if playlists are passed as command-line arguments they will be processed instead of whatever is specified in the @playlists array at the top of the script. Make sure to enclose playlist names in quotes if the playlists have spaces or special characters. For example, here's how I'd send three playlists to my connected devices on the command line:
/User/user/bin/updateipod.pl "michael best" 0pods "80's by rory"
I schedule it to run hourly with cron:
0 * * * * /Users/user/bin/updateipod.pl > /dev/null 2>&1
But you can do this if you want more output:
0 * * * * /Users/user/bin/updateipod.pl >> /Users/user/Desktop/ipod_update_log.txt 2>&1
That's it! Here's the script. Click "expand source" below to view:
#!/usr/bin/perl

# 2009-07-05
# Auto-sync specified playlists to all attached iPods/iPhones
# Specify playlist names in the @playlist array OR enter them
# as arguments (in single quotes) on the command line like this:
#
# $ ./updateipod.pl "michael best" "0pods" "80's by rory"
#
# I run this hourly with the cronjob:
#
# 0 * * * * /Users/user/bin/updateipod.pl > /dev/null 2>&1
# 
# use something like this if you want to see the output for each run:
# 
# 0 * * * * /Users/user/bin/updateipod.pl >> /Users/user/Desktop/ipod_update_log.txt 2>&1
#
# some applescript in this script was borrowed from "Selected Playlists To iPod v2.1" at the very excellent http://www.dougscripts.com/

use strict;
use POSIX qw(strftime);

my @playlists = (

 "jamz",
 "0pods",
 "punk best",
 "stoner best"
 
 );
 
# if there are playlists entered as arguments on the cmd line
# those are gonna trump whatever is specified above
if ($ARGV[0]) {
 @playlists = @ARGV;
}

my $time = strftime( "%Y-%m-%d %H:%M:%S",localtime(time));
print "syncing of playlists started at $time\n";

# check if itunes is running, start if not
checkIfRunning();

# get the sources of any attached ipods.  exit if there are none attached.
my ($id_ref,$name_ref) = getiPodSources();
my @ipod_ids = @$id_ref;
my @ipod_names = @$name_ref;
my $devices = @ipod_ids;

# exit if we have no attached ipods
if (!$ipod_ids[0]) {
 print "no manually managed iPods detected, exiting...\n";
 exit;
}

# make sure the playlists exist in the itunes lib.
my @confirmed_playlists = ();
foreach my $playlist (@playlists) {
 my $exists = checkPlaylistExists($playlist);
 if($exists) {
  push(@confirmed_playlists,$playlist);
 } else {
  print "\"$playlist\" is not a valid playlist in the iTunes library, skipping...\n";
 }
}

# make sure we have at least one valid playlist
if (!$confirmed_playlists[0]) {
 print "no valid playlists entered, exiting...\n";
 exit;
}

# loop thru our ipods first, then within that,
# loop thru our playlists array and sync them to each ipod
for (my $i=0;$i<$devices;$i++) {
 foreach my $playlist (@confirmed_playlists) {
  print "telling itunes to sync playlist \"$playlist\" to $ipod_names[$i]\n";
  my $output = syncPlaylist($ipod_ids[$i],$ipod_names[$i],$playlist);
  if ($output ne '1') {
   print "${output}, skipping...\n";
   next;
  }
 } # end loop thru playlists
} # end for thru ipods

print "done.\n\n";
exit;

#################################################################################################
###################                      SUBROUTINES                   #####################
#################################################################################################

sub stripNonAlph {
 # strips non-alphanumeric characters from strings
 
 my($text) = shift;
 $text =~ s/([^0-9a-zA-z ])//g;
 return ($text);

} # end sub strip non alph

sub checkIfRunning {
 # checks if iTunes is running
 # starts it if not
 
 my $start=`/usr/bin/osascript <<END
 --start the program if necessary
 set isRunning to 0
 tell application "System Events"
  set isRunning to ((application processes whose (name is equal to "iTunes")) count)
 end tell
 if isRunning is 0 then
  tell application "iTunes"
   activate
  end tell
 end if
 `;
 
} # end sub check if running

sub getiPodSources {
 # gets the system source IDs of any attached ipods
 # (assumes there's more than 1 in case there actually are)
 # returns in semicolon-delimited list
 
 my $source=`/usr/bin/osascript <<END
 tell application "iTunes"
  set validiPods to {}
  set myPods to ""
  set myPodsNames to ""
  set myPodReturn to ""
  set ipodSources to (get every source whose kind is iPod)
  if ipodSources is {} then
   return 0
  end if
  
  repeat with s in ipodSources
   try -- can't make a playlist on an iPod set to sync automatically, so this will fail.  
    set tp to make new user playlist at s with properties {name:("p)" & my get_temp_name() & "-^")}
    delete tp
    set end of validiPods to s
   end try
  end repeat
  
  if validiPods is {} then
   return 0
  end if
  
  repeat with i from 1 to (length of validiPods)
   set myPods to myPods & ";" & id of item i of ipodSources
   set myPodsNames to myPodsNames & ";" & name of item i of ipodSources
  end repeat
  
  set myPodsReturn to myPods & "|" & myPodsNames
  return myPodsReturn
  
 end tell
 to get_temp_name()
  return (get time of (get current date)) as text
 end get_temp_name
 END`;
 
 chomp($source);
 $source =~ /^;(.*?)\|;(.*?)$/;
 my ($id,$name) = ($1,$2);
 my @ids = split(/;/,$id);
 my @names = split(/;/,$name);
 return (\@ids,\@names);
 
} # end sub getiPodSources

sub checkPlaylistExists {
 # checks if the itunes playlist exists
 
 my $playlist = shift;
 my $exists=`/usr/bin/osascript <<END
 tell application "iTunes"
  if (exists playlist "$playlist") then
   return 1
  else
   return 0
  end if
 end tell
 END`;
 $exists = &stripNonAlph($exists);
 return ($exists);
 
} # end check if playlist exist

sub syncPlaylist {
 # takes the id of the source (ipod) and the name of the playlist to sync as arguments
 
 my ($ipod_id,$ipod_name,$playlist) = @_;
 my $sync = `/usr/bin/osascript <<END

 tell application "iTunes"
  set playlistName to "$playlist"
  set main_lib to library playlist 1
  set ipod_src to item 1 of (get every source whose id is $ipod_id)
  set ipod_lib to library playlist 1 of ipod_src
  set playlistSize to 0
  set iPodFreeSpace to free space of ipod_src
  set copySuccess to 0
  set playlistSuccess to 0
  
  set myPlaylists to (get every user playlist)
  repeat with thisPlaylist in myPlaylists
   if name of thisPlaylist is playlistName then
    
    --loop thru the playlist to determine if there is enough
    --free space on the ipod for any songs not already on it
    set eop to index of last track of thisPlaylist
    repeat with i from 1 to eop
     --determine if the song is already on ipod lib
     set i_track to track i of thisPlaylist
     if (get class of i_track) is file track then
      tell i_track to set {nom, art, alb, siz, kin, tim, pid} to {get name, get artist, get album, get size, get kind, get time, get persistent ID}
      try
       -- is track already in iPod library?  setting this will fail if not
       set new_i_track to (some track of ipod_lib whose name is nom and artist is art and album is alb and size is siz and kind is kin and time is tim)
      on error -- if not, add size of track to running total of tracks not on ipod
       set playlistSize to playlistSize + siz
      end try
     end if
    end repeat --thru selected playlist
    
    if free space of ipod_src is greater than playlistSize then
     -- establish iPod playlist
     try
      --does the iPod playlist already exist?  if so we delete it.
      --why?  because if we add additional tracks to the end of it, the track list will no be in the 
      --same order as they are on our iTunes playlist
      set new_ipodplaylist to user playlist playlistName of ipod_src
      delete new_ipodplaylist
      set new_ipodplaylist to (make new user playlist at ipod_src with properties {name:playlistName})
     on error
      -- if not, create the new iPod playlist
      set new_ipodplaylist to (make new user playlist at ipod_src with properties {name:playlistName})
     end try
     
     --I'm not checking to see if the tracks already exist on the ipod before copying
     --because it *appears* itunes won't duplicate them.  maybe this is a new feature
     try
      duplicate (every track of thisPlaylist) to ipod_lib
      set copySuccess to 1
     on error
      return "error copying tracks to iPod $ipod_name"
      set copySuccess to 0
     end try
     try
      duplicate (every track of thisPlaylist) to new_ipodplaylist
      set playlistSuccess to 1
     on error
      return "error duplicating track listing to playlist $playlist on iPod $ipod_name"
      set playlistSuccess to 0
     end try
     
     if copySuccess is 1 and playlistSuccess is 1 
      return 1
     end if
     
    else
     return "not enough free space on iPod $ipod_name to copy all tracks on $playlist"
    end if
    
   end if --current playlist = selected playlist
  end repeat
  
 end tell
 `;
 $sync = &stripNonAlph($sync);
 return ($sync);
 
} # end sub syncPlaylist

__END__
Some of the applescript was borrowed from this script at the very awesome Doug's Applescripts for iTunes.

Record iPhone conversations on a Mac with WireTap

I know what you're thinking: this seems creepy. However, there is actually a legitimate purpose here. The reason I want to do this is to record work-related conference calls to supplement my notes. Oftentimes, even with notes, it's unclear as to exactly what was discussed and decided. Disclaimer: there are state laws regarding recording of phone calls you should be aware of if you're going to do this. Google it.

The way I do this is to split the output (the sound coming from the other end of the line) from the iPhone - one signal to the earbuds so you can hear it and one to the computer's line-in input for recording. The input (your voice) is heard by the folks at the other end because there's a mic on your earbuds picking it up and sending it back to the iPhone. However, the computer can't pick that up, so I configure the recording device (in this case, WireTap) to also record what's coming in the computer's built-in mic. This way you've got both sides of the conversation available in the recording. This is one of the great things about WireTap - it can record from two sources at once.

Getting Started
There are a few things you'll need to get if you don't already have:
  • An iPhone (actually any phone that has a mini-jack in/out will work)
  • Earbuds with microphone. The ones that came with the iPhone work well. I have these and they are great. I'm sure a headset would work too.
  • A mini-jack splitter (couple bucks at RadioShack)
  • Male-to-male mini-jack cable (couple bucks at RadioShack)
  • WireTap Studio software. Despite the name, recording phone coversations is actually not what it's designed for! It's a bit spendy at $69, but you can do a free 30-day trial with no software limitations.
What You Need
Setting Up
Plug the splitter into the jack of the phone. Then plug your earbuds/headset into one of the splitter's jacks, and the direct male-to-male cable into the other. Plug the other end of the male-to-male cable into the microphone input on your mac.

Setup
Starting the Recording
Once you have the phone and cables in place, start up WireTap. if the small black controller isn't visible, go Window (menu) -> Controller to view it. In the controller window, configure the first input to be "Line In", and the second input to be "Internal Microphone". See below for example.

WireTap Controller

At this point, you may want to do a test call to be sure your levels are correct. Just hit the round "record" button on WireTap's controller and place a call (maybe to a friend or an 800 number you know will be answered by an automated system, like your bank). Be sure that you hear at least 30 seconds of sound from the other end, and make sure you talk as well to get an accurate sampling of sound from both ends.

After hanging up, hit the "stop" button on WireTap's controller, and switch to the "Library" window. (Window Menu -> Library). Your recording will probably be called something like "Built-in Input_recording". You can highlight it and play it right from that window, or you can double-click it to view in the editor. When viewing in the editor, the neat thing is that WireTap shows the sound waves for each source separately (they are color-coded). In the example below, my voice is purple while the caller's voice is grey.

Phone Sound Waves
Listen to the levels, if one side is too low or too high, you can adjust the levels. Go System Preferences -> Sound to open the Sound Settings window. The two lines you want to edit are the "Internal Microphone" and "Line In". You can even do this during your test recording so that you are seeing the actual sound in the level meters. If too low or too high, adjust slider bars. I've found checking the box "Use ambient noise reduction" for the Internal Microphone works well.

System Sound Settings
Once you have your levels right and can hear both sides of a test conversation, you're ready to record! You can export your recordings from the WireTap library as MP3s (highlight your recording and hit the "Local" button), which is good if you want to catalog elsewhere on your computer.

Update 2009-07-23: by request, here's an example of a recorded phone conversation using this method:

Use applescript and keyboard shortcuts to skip forward or back in iTunes

When I'm listening to long audio files, like podcasts or time-shifted programs, often I want to skip ahead an exact duration of time. For example, if the program has commercials, I want to skip ahead exactly 3 minutes. If I overshoot, then I want to skip back 10 seconds until I hit the program's return from break. I set up a set of applescripts that I trigger with Quicksilver keyboard shortcuts (see short how-to on adding hotkeys with quicksilver). This prevents me from having to actually go to iTunes, and manually move the time slider forward and back until I get the right position.

Here's the script for moving forward 3 minutes:
tell application "iTunes"
 if player state is playing then
  -- length of current track:
  set trackTime to duration of current track
  --get the current position time:
  set currTime to get player position
  set currSkip to currTime + 180
  
  -- checks if new position is greater than 
  -- length of track, and corrects it if it is:
  if currSkip > trackTime then
   set currSkip to trackTime
  end if
  
  set player position to currSkip --skip to new position
 end if
end tell
...and here's one for skipping back 10 seconds:
tell application "iTunes"
 if player state is playing then
  --get the current track time:
  set currTime to get player position
  if currTime < 10 then
   --go to start of the track:
   set currSkip to 0
  else
   --otherwise, skip backwards 
   set currSkip to currTime - 10
  end if
  
  set player position to currSkip --skip to new position
 end if
 
end tell
Note that the time is kept in seconds (not minutes). So to create scripts for varying amounts of time just edit the seconds and save as a separate script. Save in your user's "scripts" folder, ~/Library/Scripts/ as something like "skip3min.scpt".

Thursday, June 25, 2009

Top 10 internet radio programs

So, the difference between these and my top 10 podcasts is that these either a) are not available as podcasts, usually for music copyright reasons or 2) are available as podcasts but only if you pay a subscription fee. As such, like any good radio program, these are prime candidates to be time-shifted so you can listen when you want.

Here are my faves:
  1. Overnight Groove - Saturdays at 3-5AM PT...this happens to be when I am sleeping, so...time-shift! 70s and 80s dance music of the finest quality. Emphasis on the less hit-radio tracks, though those are peppered in for good measure. Awesome program if you like to feel great.
  2. Crap from the Past - The name says it all. DJ Ron "Boogiemonster" Gerber calls it "like a graduate-level course in pop music". The guy is an insane encyclopedia of pop music knowledge, and a real DJ in the classic sense. His brilliant intro spot says it all. And...he loves Yacht Rock.
  3. Quiet Space - For ambient music lovers, this is the best-curated program available on the internet. DJ Paul Gough (aka the artist pimmon) plays an hour's-worth of the best ambient he can find, both old and new, twice a week.
  4. Moneytalk - Bob Brinker's a ruthless old codger bent on belittling politicians and even listeners he thinks are stupid (which is alot of them). He has a 3-hour call-in show on Saturdays and Sundays (I use KGO's stream which is very reliable). Best part of the show is the special guest, who is featured in the last hour of each show. Only downside is that the commercials are frequent and annoying. But, as long as the program is time-shifted, you can just skip over them.
  5. Sonic Reducer - The best punk/hardcore/garage rock show on the internets. 3 hours of hard rock bliss every Saturday night. Gets its name from a song by early Cleveland punk band The Dead Boys, a favorite of Anthony Bourdain. How's that for a tangential anecdote.
  6. Philosophy Talk - "The show that questions everything...except your intelligence." I love this program because these guys, professors of Philosophy at Stanford, have great radio personalities and make complex subjects understandable. Not every show is a hit, but most are.
  7. Sleepbot Environmental Broadcast - This isn't a radio program per se, as it's always on and streaming, 24 hours a day. But I time-shift it because while most of what the curator (Dan Foley) plays is great, sometimes his tastes and my own diverge...and I like to skip over those tracks. I record about 3 hours of this nightly so I can listen during the day while I work. I've found alot of great music on this station.
  8. The Next Big Thing - This show no longer airs, unfortunately. It was on select public radio stations from 1999 to 2005. It's on this list because there is a streaming archive (but no podcast, strangely) that features the vast majority of the shows. The show was a fine example of radio documentary, produced by WNYC. I don't know why it ended, but the WNYC "radio doc" slot seems to have been filled by RadioLab, another fine program. Anyway, see my post about recording "The Next Big Thing" archived shows.
  9. Surface Noise - DJ Lounge Laura on WMNF in Tampa - and myself - have compatibly eclectic tastes in rock. Her starting point is (roughly) punk in the 80s, but she takes wide detours from that format...glam, classic rock, pop, whatever. Like Ron "Boogiemonster" Gerber, she's a DJ in the classic sense of the term. Only downside is that the sound quality of the stream is pretty low, but not so much so that it prevents me from listening.
  10. The Dave Ramsey Show - I am a Dave Ramsey devotee. I don't agree with all his views, but in terms of opinions on personal finance (it's actually more about personal responsibility), he's dead on. He's not afraid to lay the smackdown on his listeners, but they tend to deserve it. A little tough love is good every now and then.

Top 10 podcasts

I listen to ALOT of podcasts. Not that anyone should care, but these are my top 10 podcasts. Part of my continuing series of "top tens". Obviously very heavily weighted towards public radio programs...but I am not a socialist.
  1. This American Life - Still the best hour of info-tainment(?) docu-tainment(?) available!
  2. Marketplace - I listen to this every day.
  3. NPR's Planet Money - Sort of like a "This American Life" but only concerned with money/finance and it's 3x a week. What I love about it is it translates all the financial gibberish that's in the news into something I can understand. It's also relatively politically un-biased (at least in terms of finance) for NPR.
  4. Stuff You Should Know - Josh Clark and Chuck Bryant are a couple of the funniest info-tainers(?) on the internets! I learn about random things like how to survive plane crashes, how lobotomies work, and so forth . Twice a week. Nice, Chuck.
  5. EconTalk - Since the financial meltdown everyone's become an amateur economist, myself included. This show gives me just enough of an education to be dangerous. What I like about EconTalk is that the host, Russ Roberts, is very fiscally conservative but isn't afraid to have guests on who have vastly different opinions than him. Makes for great conversation.
  6. American RadioWorks - 60 minutes, one subject explored in-depth. Basically, it's like "Frontline" for radio. VERY high-quality documentaries. It would be closer to the top on my list, but they are not that frequent. I think it's because they are so well-made they take a while to produce.
  7. RadioLab - Hard to really describe this show. It's documentary, science, drama and artful sound design rolled into one. It's beautiful and captivating. Here, I'll let one of the show's hosts, Jad Abumrad, explain:
  8. The Moth - Bringing back the art of storytelling! The podcast is sort of a "best of" stories, chosen from years worth of archives.
  9. Cosmic Slop - A collection of not-oft-heard tracks, mostly from the 70s and 80s. Chuck and Joel are great radio personalities, and have encyclopaedic knowledge of popular (and in this case, not-so-popular) music. These guys have intorduced me to alot of real gems. I owe my love of Joan Armatrading to them. Great stuff, I just wish it was more frequent.
  10. BBC Radio Documentaries - Selected radio docs that run on BBC throughout the week. Usually 20-25 mins long. Can be hit-or-miss, but I have heard some really great ones.

Record WNYC's "The Next Big Thing" streams to MP3

Remember that show "The Next Big Thing" in WNYC? Good show. Ran from 1999 to early 2006. I miss it. Fortunately, they have most of the shows archived on the WNYC site. Unfortunately, they are in realaudio. When I tried to listn to the realaudio stream on my mac with the crappy realaudio player (are they even still in business?? If so, why?), I got a random collection of clicks and blips.

But I soon found out that mplayer will play them. So I wrote up a little script that will rip the realaudio stream to MP3, which is better anyway because then I can listen on my iPod. The quality isn't great - I think it's a 32kbps stream, but that's roughly equivalent to AM radio quality. It's listenable. It will run on any unix-based system (like a mac) with the following pre-requisites:
  • mplayer
  • wget
  • lame
  • sox
  • Perl 5.8.8+
  • Perl Module: MP3::Tag
Note: you may need mplayer configured with realaudio codecs as per this article, but I tried it on a linux system that didn't have them, so it should work without.

One caveat is that although mplayer has the capability to dump realaudio streams very quickly (by adding a large value, like 5000000 to the -bandwidth switch), I found that for some reason alot of audio in these streams was skipped. Not good. So I left out the bandwidth switch and let it dump the stream in real time. Takes alot longer, but you get all the audio.

Click "expand source" to view the code below. I call it "nextbigthing.pl":
#!/usr/bin/perl

# 2009-06-21
# created to take a URL of a WNYC's "The Next Big Thing" show archives as input (first argument on cmd line)
# ex: $ ./nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/05
# and create tagged MP3s of each show on the page as output
# 
# PREREQs
# * mplayer with links to realaudio codecs (see here: http://www.macosxhints.com/article.php?story=20050130184054216)
# * wget
# * lame
# * sox

use strict;
use POSIX qw(strftime);
use MP3::Tag;

# config these
my $wget = "/usr/bin/wget";
my $mplayer = "/usr/bin/mplayer";
my $lame = "/usr/bin/lame";
my $sox = "/usr/bin/sox";
my $tmpfolder = "/home/user/tmp/tnbt/"; # create this if it doesn't exist
my $outfolder = "/home/user/Music/the_next_big_thing/"; # create this if it doesn't exist

my @rmfiles = ();

# get the url off the command line
if (!$ARGV[0]) {
 print "no URL found.  put the url in single quotes as the first argument after calling this program on the command line\n";
 exit;
} 

# wget the page
my $outfile = $tmpfolder . "tmpout.txt";
my $wout = `$wget -U 'Mozilla/5.0 (Windows; U; Windows NT 5.1) Gecko/20041001' --timeout=30 -t 2 -q --output-document=${outfile} $ARGV[0]`;
if (not(-e $outfile)) {
 myExit("nothing found. check url and try again",\@rmfiles);
} else {
 push(@rmfiles,$outfile);
}

# put file into array
open(FILE,"$outfile");
 my @lines = <FILE>;
close(FILE);

# search the array for keywords and place our show info into a hash
my %SHOWS = ();
my $counter=0;
my ($showdate,$episode_name,$episode_date,$episode_description);
foreach my $line (@lines) {
 chomp($line);
 
 # there's a really specific format we're looking for here.  if they change their template, this won't work at all
 # we're looking for show info here
 if ($line =~ /episodenamesmall/) {
  $lines[$counter] =~ /^.*?episodes\/(\d\d\d\d)\/(\d\d)\/(\d\d)">$/;
  $showdate = "${1}-${2}-${3}";
  $lines[$counter+1] =~ /^(.*?)</;
  $episode_name = $1;
  $lines[$counter+2] =~ /^<.*?>(.*?)<.*?>$/;
  $episode_date = $1;
  $lines[$counter+3] =~ /^<p>(.*?)<\/p>$/;
  $episode_description = $1;
  
  $SHOWS{$showdate}{'showdate'} = $showdate;
  $SHOWS{$showdate}{'episode_name'} = $episode_name;
  $SHOWS{$showdate}{'episode_date'} = $episode_date;
  $SHOWS{$showdate}{'episode_desc'} = $episode_description;
  
  
 } # end search for 'episodenamesmall'

 # we're looking for the url to the realmedia here
 if ($line =~ /<a class="listen"/) {
  my $rafiles;
  my @ramfiles = ();
  $lines[$counter+2] =~ /^\s+href="\/stream\/ram.py\?(.*?)" class.*?$/;
  $rafiles = $1;
  $rafiles =~ s/file[\d]{0,2}=//g;
  my @tmpfiles = split(/&/,$rafiles);
  foreach my $tmpfile (@tmpfiles) {
   my $fullurl = 'rtsp://raudio.wnyc.org/' . $tmpfile;
   push(@ramfiles,$fullurl);
  }
  my $print_ra = join("^",@ramfiles);
  @{ $SHOWS{$showdate}{'files'} } = @ramfiles;

 } # end search for 'listen'
 
 $counter++;
 
} # end foreach loop thru html file

# loop thru the hash, grab each show, and encode
for my $show ( sort keys %SHOWS ) {
 my @wavfiles = ();
 
 print "Episode $SHOWS{$show}{'episode_name'} - $SHOWS{$show}{'episode_date'}\n";
 
 # our filenames
 my $wavfilename = ${tmpfolder} . replaceSpace(stripChars($SHOWS{$show}{'episode_name'})) . '_' . $SHOWS{$show}{'showdate'} . '.wav';
 my $mp3 = $wavfilename;
 $mp3 =~ s/${tmpfolder}/${outfolder}/;
 $mp3 =~ s/\.wav/\.mp3/;
 # if an mp3 already exists in the dest. dir, skip
 if (-e $mp3) {
  print "\nFilename $mp3 already exists, skipping...\n\n";
  next;
 }
 
 foreach my $file (@{ $SHOWS{$show}{'files'} }) {
  #print "File: $file\n";
  $file =~ /^rtsp:\/\/raudio.wnyc.org\/nbt\/(.*?)$/;
  my $ra_name = $1;
  # dump the raw .ra file
  my $raw = "${tmpfolder}${ra_name}";
  # removed the bandwidth switch because although it speeds up the dumping, it seems to skip alot of audio.
  # so as of now it's just dumping in real-time
  #my $get_cmd = "$mplayer -bandwidth 5000000 -noframedrop -dumpfile $raw -dumpstream '$file'";
  my $get_cmd = "$mplayer -noframedrop -dumpfile $raw -dumpstream '$file'";
  print "MPLAYER DUMP COMMAND: $get_cmd\n";
  my $get_out = `$get_cmd`;
  # convert files to wav
  my $filename = "${tmpfolder}${ra_name}.wav";
  my $mplayer_cmd = "$mplayer $raw -vc dummy -vo null -af volume=0,channels=2 -ao pcm:waveheader:file=$filename";
  print "MPLAYER WAVE COMMAND: $mplayer_cmd\n";
  my $mp_out = `$mplayer_cmd`;
  if (-e $filename) {
   push(@rmfiles,$filename);
   push(@rmfiles,$raw);
   push(@wavfiles,$filename);
  } else {
   myExit("no wavfile was created from $mplayer_cmd",\@rmfiles);
  }

 } # end foreach thru wavfiles
 
 # cat wavfiles together with sox
 my $wavfilejoin = join(' ',@wavfiles);
 my $soxcmd = "(for wavfile in ${tmpfolder}*.wav; do $sox \"\$wavfile\" -t .raw -r 44100 -sw -c 2 -; done) | $sox -t .raw -r 44100 -sw -c 2 - -t .wav $wavfilename";
 print "SOX CONCAT COMMAND: $soxcmd\n";
 my $soxout = `$soxcmd`;

 # encode the wav file
 if (-e $wavfilename) {
  push(@rmfiles,$wavfilename);
  my $title = "The Next Big Thing: \"" . stripChars($SHOWS{$show}{'episode_name'}) . "\" (" . $SHOWS{$show}{'episode_date'} . ")";
  my $author = "Dean Olsher";
  my $composer = "WNYC";
  my $album = "The Next Big Thing";
  $SHOWS{$show}{'showdate'} =~ /^(\d\d\d\d)-(\d\d)-(\d\d)$/;
  my $year = $1;
  my $info_url = $ARGV[0];
  my $genre = "Talk";
  # caused some problems adding the ID3v2 stuff on cmd line so we'll just have MP3::Tag do it
  #my $id3 = " --add-id3v2 --ignore-tag-errors --tt \"$title\" --ta \"$author $composer\" --tl \"$album\" --ty \"$year\" --tc \"$info_url\" --tg \"$genre\"";
  #my $encode = "$lame -V0 -h -b 128 --quiet --vbr-new$id3 $wavfilename $mp3";
  my $encode = "$lame -V0 -h -b 128 --quiet --vbr-new $wavfilename $mp3";
  print "LAME ENCODE COMMAND: $encode\n";
  my $enc_out = `$encode`;
  
  # add proper ID3 tags and clean up
  if($mp3) {
   my $comments = stripChars($SHOWS{$show}{'episode_desc'});
   &id3v2tag($mp3,$title,$year,$author,$album,$info_url,$comments,$genre,"lame");
   myExit("SUCCESS! Encoding complete for $title",\@rmfiles,1);
  } else {
   myExit("the mp3 doesn't exist so I can't tag it",\@rmfiles);
  } # end if for mp3 exists
  
 } else {
  myExit("our resampled wav file doesn't exist so I can't encode an mp3",\@rmfiles);
 } # end if for wav exists
 
 print "\n\n";
 
} # end foreach thru shows

print "Done\n";
exit;

##################### SUBS ###########################

sub myExit {
 
 my ($message,$rmfiles_ref,$donotexit) = @_;
 my @rms = @$rmfiles_ref;
 foreach my $remove (@rms) {
  if (-e $remove) {
   print "REMOVING $remove\n";
   unlink($remove)
  }
 }
 print "$message\n";
 if (!$donotexit) {
  exit;
 }
 
} # end sub myExit

sub stripChars {

 my($text) = @_;
 
 $text =~ s/\n/ /g; # strip carraige returns
 $text =~ s/\t/ /g; # strip tabs
 $text =~ s/\a/ /g; # strip carraige returns
 $text =~ s/"/'/g; # strip quotes and replace with single quotes
 $text =~ s/\s+/ /g; # strip repeating spaces and replace with one
 return ($text);

} # end sub stripchars

sub replaceSpace {

 my($text) = shift;
 $text =~ s/([^\w+\s+])//g;
 $text =~ s/^\s+//;
 $text =~ s/\s+$//;
 $text =~ s/([\s+])/_/g;
 return ($text);

} # end sub replacespace

sub id3v2tag {
 # adds id3v2 tag to mp3s

 my ($file,$title,$year,$author,$album,$info_url,$comments,$genre,$encoder) = @_;
 my $mp3 = MP3::Tag->new($file);
 $mp3->get_tags();
 $mp3->new_tag("ID3v2");
 $mp3->{ID3v2}->add_frame("TALB", "$album");
 $mp3->{ID3v2}->add_frame("TIT2", "$title");
 $mp3->{ID3v2}->add_frame("TPE1", "$author");
 $mp3->{ID3v2}->add_frame("TCON", "$genre");
 $mp3->{ID3v2}->add_frame("TSSE", "$encoder");
 $mp3->{ID3v2}->add_frame("TYER", "$year");
 $mp3->{ID3v2}->add_frame("COMM", "ENG", "", "$comments $info_url");
 $mp3->{ID3v2}->add_frame("TIT3", "$comments $info_url");
 $mp3->{ID3v2}->add_frame("TRSN", "$comments $info_url");
 $mp3->{ID3v2}->add_frame("TXXX", "$comments $info_url");
 $mp3->{ID3v2}->add_frame("WORS", "$comments $info_url");
 $mp3->{ID3v2}->add_frame("WXXX", "$comments $info_url");
 $mp3->{ID3v2}->write_tag;
 $mp3->close();

} # end sub id3v2tag



The neat thing about this is it will retrieve the show info from the archive web page and add it to the ID3 info, so the MP3s are named and dated properly. It's called with the URL of each archive page like this:
./nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/05
I created a wrapper script that sends a whole bunch of the URLs to the script like this:
#/bin/bash

/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/06;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/07;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/08;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/09;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/10;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/11;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2002/12;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2003/01;
/home/user/bin/nextbigthing.pl http://www.wnyc.org/shows/tnbt/episodes/2003/02;
Then just let it run until all the streams are recorded!

Script for time-shifting internet radio programs (aka "Streamshifter")

There are alot of time-shifting internet radio stream recorders out there (i.e. TiVo for internet radio) like RadioShift and WireTap Studio, and I've tried them all. Inevitably I've run into limitations with them. Either they are unreliable (you go to listen to your program and find it didn't record and there's no clear reason why), they are memory hogs, or they can't be run from a server (they are gui programs mant to be run locally). So, I am in the process of writing my own program in perl. As of now it is very functional and feature-rich. As far as I can tell, it's the only stream recorder that can be run from a server (advantageous because the bandwidth is more reliable than a home connection) and administered from the web.

I encourage you to try it out on some unix-based system (I'm on a mac) and give me feedback. I'm always working to improve it. There's not much documentation at this time (just this post), but if you're a script hacker it's pretty easy to figure out. You'll need:
  • Perl 5.8.8 or higher (though earlier versions may work)
  • MySQL 4.1.22 or higher
  • mplayer
  • sox
  • lame
  • wget
  • Perl Modules: DBI, CGI, POSIX, MP3::Info, MP3::Tag
  • Optional: Apache w/PHP with phpMyAdmin
I've tried to keep everything packed into one large script for ease of installation, but there are a couple additional things: the SQL, a small config file, and a cronjob.

NOTE: To keep the vertical scrolling down I've collapsed the source of the longer files below. Click "expand source" to view.

Here's the SQL with test data. I intend to build a web-based interface for this at some point, but for now I just run apache and admin it with phpMyAdmin. I created a separate database called "streamshifter" and imported this SQL into it:
CREATE TABLE `streamshifter_config` (
  `id` int(5) NOT NULL AUTO_INCREMENT,
  `program_name` varchar(24) NOT NULL DEFAULT 'streamshifter.cgi',
  `your_timezone` enum('AKT','AT','CT','ET','HT','MT','NT','PT','YT') NOT NULL DEFAULT 'ET' COMMENT 'Program scheduling times must be for this timezone.  Adjustments will be made automatically when traveling based on computer timezone.',
  `id3_timestamp_format` varchar(36) NOT NULL DEFAULT '' COMMENT 'If you want the timestamp included in the ID3 tags, please enter the format here.  Leave empty if you do not want the timestamp in the ID3 name.  Use MySQL anchors, see this for reference: http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.htm',
  `path_to_cgibin` varchar(128) NOT NULL DEFAULT '',
  `path_to_library` varchar(255) NOT NULL DEFAULT '' COMMENT 'Full path to the library where you want to store recorded files.  Use trailing slash.',
  `add_to_itunes` enum('Yes','No') NOT NULL DEFAULT 'No' COMMENT 'Select "yes" if you''d like streamshifter to add the finished track to your itunes library.  This works *only* on Mac OS X, please select No for all other operating systems.',
  `auto_create_subdirs` enum('Yes','No') NOT NULL DEFAULT 'Yes' COMMENT 'Do you want it to auto-create subdirectories  in your library for recorded files?',
  `path_to_desktop` varchar(255) NOT NULL DEFAULT '' COMMENT 'Full path to your desktop, which is the default save location and where test files will be placed.  use trailing slash.',
  `path_to_rawdata` varchar(255) NOT NULL DEFAULT '' COMMENT 'Full path to temp file directory.  Temp files will be large, but will be removed after recording.  Use trailing slash.',
  `path_to_mplayer` varchar(128) NOT NULL DEFAULT '' COMMENT 'mplayer is required',
  `path_to_sox` varchar(128) NOT NULL DEFAULT '' COMMENT 'sox is required',
  `path_to_lame` varchar(128) NOT NULL DEFAULT '' COMMENT 'lame is required',
  `path_to_normalize` varchar(128) NOT NULL DEFAULT '' COMMENT 'normalize is required',
  `path_to_wget` varchar(128) NOT NULL DEFAULT '' COMMENT 'wget is required',
  `path_to_sendmail` varchar(128) DEFAULT NULL COMMENT 'If you want StreamShifter to send emails when there are new shows or errors with recording. ',
  `path_to_nice` varchar(128) DEFAULT NULL COMMENT 'The path to ''nice'' on your system.  CPU-intensive processes like lame, sox and normalize are niced.  Leave empty if you don''t want to use.',
  `max_recording_hours` int(2) NOT NULL DEFAULT '7',
  `timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;

-- 
-- Dumping data for table `streamshifter_config`
-- 

INSERT INTO `streamshifter_config` VALUES (1, 'streamshifter.cgi', 'PT', '%W, %M %e %h:%i %p %Z', '/User/user/bin/', '/Users/user/Music/Import/', 'Yes', 'Yes', '/Users/user/Desktop/', '/Users/user/tmp/', '/opt/local/bin/mplayer', '/opt/local/bin/sox', '/opt/local/bin/lame', '/opt/local/bin/normalize', '/opt/local/bin/wget', '/usr/sbin/sendmail', '/usr/bin/nice', 5, '2009-11-05 21:31:00');



CREATE TABLE `streamshifter_programs` (
  `id` int(11) NOT NULL auto_increment,
  `schedule_id` int(5) NOT NULL default '0',
  `title` varchar(255) NOT NULL default '',
  `recording_date` datetime NOT NULL default '0000-00-00 00:00:00',
  `recording_length` time NOT NULL default '00:00:00',
  `bitrate` int(4) NOT NULL default '0',
  `filesize` float NOT NULL default '0',
  `path_to_file` varchar(255) NOT NULL default '',
  `info_url` varchar(255) NOT NULL default '' COMMENT 'Can be a playlist',
  `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `path_to_file` (`path_to_file`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

CREATE TABLE `streamshifter_schedule` (
  `id` int(11) NOT NULL auto_increment,
  `user_id` int(3) NOT NULL default '0',
  `title` varchar(152) NOT NULL default '',
  `author` varchar(152) NOT NULL default '',
  `album_or_episode` varchar(152) default NULL,
  `composer_or_network` varchar(152) default NULL,
  `genre` enum('A Cappella','Acid','Acid Jazz','Acid Punk','Acoustic','Alternative','Alt. Rock','Ambient','Anime','Avantgarde','Ballad','Bass','Beat','Bebob','Big Band','Black Metal','Bluegrass','Blues','Booty Bass','BritPop','Cabaret','Celtic','Chamber Music','Chanson','Chorus','Christian Gangsta Rap','Christian Rap','Christian Rock','Classical','Classic Rock','Club','Club-House','Comedy','Contemporary Christian','Country','Crossover','Cult','Dance','Dance Hall','Darkwave','Death Metal','Disco','Dream','Drone','Drum & Bass','Drum Solo','Duet','Easy Listening','Electronic','Ethnic','Eurodance','Euro-House','Euro-Techno','Fast-Fusion','Folk','Folklore','Folk/Rock','Freestyle','Finance','Funk','Fusion','Game','Gangsta Rap','Goa','Gospel','Gothic','Gothic Rock','Grunge','Hardcore','Hard Rock','Heavy Metal','Hip-Hop','House','Humour','Indie','Industrial','Instrumental','Instrumental Pop','Instrumental Rock','Jazz','Jazz+Funk','JPop','Jungle','Latin','Lo-Fi','Meditative','Merengue','Metal','Musical','National Folk','Native American','Negerpunk','New Age','New Wave','News','Noise','Oldies','Opera','Other','Politics','Polka','Polsk Punk','Pop','Pop-Folk','Pop/Funk','Porn Groove','Power Ballad','Pranks','Primus','Progressive Rock','Psychedelic','Psychedelic Rock','Punk','Punk Rock','Rap','Rave','R&B','Reggae','Retro','Revival','Rhythmic Soul','Rock','Rock & Roll','Salsa','Samba','Satire','Showtunes','Ska','Slow Jam','Slow Rock','Sonata','Soul','Sound Clip','Soundtrack','Southern Rock','Space','Speech','Sports','Swing','Symphonic Rock','Symphony','Synthpop','Talk','Tango','Techno','Techno-Industrial','Terror','Thrash Metal','Top 40','Trailer','Trance','Tribal','Trip-Hop','Vocal') default NULL,
  `output_bitrate` enum('32','64','96','128','160','192','256','320') NOT NULL default '96',
  `adjust_url` enum('0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23') NOT NULL default '0' COMMENT 'The number of hours to adjust the date set in the date_formatted_url (to account for timezone differences)',
  `adjust_url_interval` enum('Future','Past') NOT NULL default 'Future' COMMENT 'If you want the date formatted url to be adjusted, need to specify whether it should be adjusted in the future or past.',
  `url` varchar(255) NOT NULL default '' COMMENT 'If the URL is date-specific and you want it to update on the day the program is scheduled, use date specifiers in place of the date/time.  See: http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html#function_date-format',
  `adjust_info_url` enum('0','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','16','17','18','19','20','21','22','23') NOT NULL default '0' COMMENT 'The number of hours to adjust the date set in the date_formatted_info_url (to account for timezone differences)',
  `adjust_info_url_interval` enum('Future','Past') NOT NULL default 'Future' COMMENT 'If you want the date formatted info url to be adjusted, need to specify whether it should be adjusted in the future or past.',
  `info_url` varchar(255) default NULL COMMENT 'Information about the program, or a playlist.  If the URL is date-specific (like, a playlist from a certain show) and you want it to update on the day the program is scheduled, use date specifiers in place of the date/time.  See: http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html#function_date-format',
  `repeat` enum('Once','Daily','Weekdays','Weekends','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday') NOT NULL default 'Daily',
  `hour` enum('00','01','02','03','04','05','06','07','08','09','10','11','12','13','14','15','16','17','18','19','20','21','22','23') NOT NULL default '00',
  `minute` enum('00','01','02','03','04','05','06','07','08','09','10','11','12','13','14','15','16','17','18','19','20','21','22','23','24','25','26','27','28','29','30','31','32','33','34','35','36','37','38','39','40','41','42','43','44','45','46','47','48','49','50','51','52','53','54','55','56','57','58','59') NOT NULL default '00',
  `start_date` date NOT NULL default '0000-00-00',
  `duration_minutes` int(6) NOT NULL default '60',
  `save_to_library` enum('Yes','No') NOT NULL default 'Yes' COMMENT 'Save recorded file to library?  If no, will be saved to desktop.',
  `episodes_to_keep` enum('All','1','2','3','4','5','6','7','8','9','10','11','12','13','14','15','20','25','30','40') NOT NULL default '5',
  `add_to_itunes_playlists` varchar(255) NOT NULL default '' COMMENT 'If on a Mac with iTunes installed, you can specify what playlists to add this track to.  Separate multiple playlists with commas.  If a specified playlist doesn''t exist, it will be created.  Ignore this field if not on a Mac.',
  `itunes_bookmarkable` enum('Yes','No') NOT NULL default 'Yes' COMMENT 'Set to "Yes" if you''re on a mac and you want streamshifter to set the track to bookmarkable (iTunes remembers the last playback position.  Handy for long programs.  Works only on Mac OS X.  Set to "No" for other operating systems.',
  `last_run` datetime NOT NULL default '0000-00-00 00:00:00',
  `last_run_result` enum('Success','Fail') default NULL,
  `last_run_comments` text,
  `active` enum('Yes','No') NOT NULL default 'Yes',
  PRIMARY KEY  (`id`),
  KEY `episodes_to_keep` (`episodes_to_keep`),
  KEY `user_id` (`user_id`),
  KEY `repeat` (`repeat`,`hour`,`minute`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=6 ;

INSERT INTO `streamshifter_schedule` VALUES (1, 1, 'This American Life', 'Ira Glass', '', 'WUIS', 'Other', '128', '0', 'Future', 'http://wuis.streamguys.net/wuis64.m3u', '0', 'Future', 'http://www.thislife.org', 'Friday', '17', '00', '2008-07-01', 60, 'Yes', '3', '0pods,this american life', 'Yes', '2009-06-19 18:02:53', 'Success', 'New episode of This American Life - Friday, June 19 05:00 PM PDT successfully recorded.  Check it out!\n\nFile: /Home/user/Music/Import/this_american_life/This_American_Life_2009-06-19-17-00-01.mp3\n\nDuration: 01:00:03', 'Yes');
INSERT INTO `streamshifter_schedule` VALUES (2, 1, 'Variations', 'DJG', NULL, 'KBCS', 'Reggae', '128', '0', 'Future', 'http://ophanim.net/cast/tunein.php/kbcs96/playlist.pls', '0', 'Future', NULL, 'Friday', '23', '00', '0000-00-00', 120, 'Yes', '5', '', 'Yes', '2009-06-20 01:07:16', 'Success', 'New episode of Variations - Friday, June 19 11:00 PM PDT successfully recorded.  Check it out!\n\nFile: /Home/user/Music/Import/variations/Variations_2009-06-19-23-00-01.mp3\n\nDuration: 02:00:41', 'Yes');
INSERT INTO `streamshifter_schedule` VALUES (3, 1, 'Philosophy Talk', 'Ken Taylor & John Perry', '', 'KALW', 'Talk', '128', '0', 'Future', 'http://www.kalw.org/media/Streaming%20Links/kalw-mp3.pls', '0', 'Future', '', 'Sunday', '10', '07', '0000-00-00', 53, 'Yes', '5', '0pods,philosophy talk', 'Yes', '2009-06-21 11:02:59', 'Success', 'New episode of Philosophy Talk - Sunday, June 21 10:07 AM PDT successfully recorded.  Check it out!\n\nFile: /Home/user/Music/Import/philosophy_talk/Philosophy_Talk_2009-06-21-10-07-01.mp3\n\nDuration: 00:53:45', 'Yes');
INSERT INTO `streamshifter_schedule` VALUES (4, 1, 'Evan "Funk" Davies Show', 'Evan Davies', '', 'WFMU', 'Other', '128', '9', 'Past', 'http://mp3archives.wfmu.org/archive/kdb/mp3jump.mp3/0:8:41/0/ED/ed%y%m%d.mp3', '0', 'Future', 'http://wfmu.org/playlists/ED', 'Wednesday', '06', '00', '0000-00-00', 180, 'Yes', '3', 'radio', 'Yes', '2009-06-24 06:05:08', 'Success', 'New episode of Evan ''Funk'' Davies Show - Wednesday, June 24 06:00 AM PDT successfully recorded.  Check it out!\n\nFile: /Home/user/Music/Import/evan_funk_davies_show/Evan_Funk_Davies_Show_2009-06-24-06-00-01.mp3\n\nDuration: 03:07:18', 'Yes');
INSERT INTO `streamshifter_schedule` VALUES (5, 1, 'Stochastic Hit Parade', 'Bethany Ryker', '', 'WFMU', 'Other', '128', '11', 'Past', 'http://mp3archives.wfmu.org/archive/kdb/mp3jump.mp3/0:8:7/0/HP/hp%y%m%d.mp3', '0', 'Future', 'http://wfmu.org/playlists/HP', 'Monday', '06', '00', '0000-00-00', 120, 'Yes', '5', 'radio,stochastic hit parade', 'Yes', '2009-06-22 06:05:09', 'Success', 'New episode of Stochastic Hit Parade - Monday, June 22 06:00 AM PDT successfully recorded.  Check it out!\n\nFile: /Home/user/Music/Import/stochastic_hit_parade/Stochastic_Hit_Parade_2009-06-22-06-00-01.mp3\n\nDuration: 02:07:52', 'Yes');

CREATE TABLE `streamshifter_users` (
  `id` int(5) NOT NULL auto_increment,
  `username` varchar(24) NOT NULL default '',
  `password` varchar(24) NOT NULL default '',
  `name` varchar(128) NOT NULL default '',
  `email` varchar(128) NOT NULL default '',
  `user_type` enum('admin','user') NOT NULL default 'user',
  `email_alerts` enum('Yes','No') NOT NULL default 'Yes' COMMENT 'Select "Yes" if you want to be alerted of new StreamShift recordings or about errors with recording',
  `timestamp` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`),
  KEY `username` (`username`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=2 ;

INSERT INTO `streamshifter_users` VALUES (1, 'user', 'yourpassword', 'Your Name', 'you@you.com', 'admin', 'Yes', '2008-08-06 08:00:30');
There's a small configuration file which the script needs. Because it will contain MySQL connection info (edit the file with your specific info) you're going to want to put it in a safe place with restrictive permissions. I call the file "streamshifter.config":
# MYSQL database connection information
MYSQL_HOST: localhost
MYSQL_DATABASE: streamshifter 
MYSQL_USER: user
MYSQL_PASSWORD: password

# LOGGING: 0=Off 1=On
LOGGING: 1

# admin's email address
WEBMASTER: you@you.com
The only editing of the script itself you need to do is edit the path to the configuration file. And chmod 755 it of course. I call it "streamshifter.cgi":
#!/usr/bin/perl

my $path_to_config = '/Applications/streamshifter.config';

use DBI;
use CGI;
use strict;
use POSIX qw(strftime);
require MP3::Info;
require MP3::Tag;

# get the streamshifter configuration
my @conf_errors = ();
my ($host,$database,$user,$mysqlpassword,$webmaster,$logging,$conf_errors_ref) = &getDbConfig($path_to_config);
if ($conf_errors_ref) { @conf_errors = @$conf_errors_ref; }
# errors, at this point, are fatal
if ($conf_errors[0]) {
 my $error = join(". ",@conf_errors);
 print "ERRORS: $error.\n";
 exit 1;
}

# in our temp dir, we need to clean up files older than x days
# there shouldn't be any, but just in case there was a crash
my $max_file_age = 2;

# set up the db connection
my $dbh = DBI->connect("DBI:mysql:database=$database;host=$host","$user","$mysqlpassword",{'RaiseError'=>1}) or die "Unable to connect: $DBI::errstr\n"; 

# see if there are arguments...(flags on the command line)
my ($errors_ref,$spec_id,$action,$rec_url,$rec_minutes) = "";
my @errors = ();
if ($ARGV[0]) {
 ($errors_ref,$spec_id,$action,$rec_url,$rec_minutes) = checkFlags(\@ARGV,$dbh);
 if ($errors_ref) { @errors = @$errors_ref; }
 if ($errors[0]) {
  my $error = join(". ",@errors);
  print "ERRORS: $error.\n";
  exit 1;
 }
}

# get more configuration info, from the db
my @config_errors = ();
my ($self,$your_timezone,$id3_timestamp_format,$script_name,$path_to_cgibin,
 $path_to_library,$add_to_itunes,$auto_create_subdirs,$path_to_desktop,$path_to_rawdata,
 $mplayer,$sox,$normalize,$lame,$wget,$sendmail,$nice,$max_recording_hours,$config_errors_ref) = &get_config($dbh);
if ($config_errors_ref) { @config_errors = @$config_errors_ref; }
if ($config_errors[0]) {
 my $error = join(". ",@config_errors);
 print "ERRORS: $error.\n";
 exit 1;
}

# get our date vars - we start the recording 1 minute earlier so we can test the url and throw an error if it fails
my ($date,$hour,$minute,$day,$year,$ts,$ts2,$date_short,$timezone) = &getDate($your_timezone);

# logfile
my $logfile_name = 'radio_log_' . $date . '.txt'; #do not change this filename!
my $logfile = $path_to_rawdata . $logfile_name;
my $delete_after = 2; # delete old logfiles older than this number of days

# check if the logfile needs rotated
&rotateFile($path_to_rawdata,$delete_after);



# the 'check' flag performs our routine maintenance and checks for scheduled programs
if ($action eq 'check') {
 
 # check for new programs, do routine maintenance.  we don't return from this
 #&log($spec_id,"check","checking",$logfile,$logging);
 &checkSchedule($self,$max_file_age,$path_to_rawdata,$max_recording_hours,$hour,$minute,$day,$date,$path_to_library,$auto_create_subdirs,$add_to_itunes,$dbh);
 
} elsif (($action eq 'record') || ($action eq 'test')) {

 # load up the program's info
 my ($path,$temp_file,$final_file,$temp_filepath,$file_filepath,
  $final_filepath,$ext,$user_id,$title,$author,$album,$composer,$genre,$url,
  $adjust_url,$adjust_url_interval,$info_url,$adjust_info_url,
  $duration_min,$duration_sec,$output_bitrate,$add_to_itunes_playlists,$itunes_bookmarkable,$adjust_url,$username,
  $contact_name,$contact_email,$email_alerts) = &getProgramInfo($spec_id,$action,$path_to_library,$path_to_desktop,$rec_url,$rec_minutes,$id3_timestamp_format,$dbh);

 # figure out what kind of url this is
 my ($success,$khz,$cmd,$get_prog,$temp_filepath,$ext,$failreason) = &urlGetMethod($url,$temp_filepath,$mplayer,$nice,$wget,$ext,$path_to_rawdata);
 if ($success == 0) {
  my $message = "\n\nCould not retrieve data from URL $url.\n\n$failreason";
  &log($spec_id,$title,$message,$logfile,$logging);
  &fail($spec_id,$temp_filepath,"$message");
  if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
  exit 1;
 } else {
  &log($spec_id,$title,"$url successfully tested",$logfile,$logging);
 }
 
 # fork for the recording process
 my $pid = fork();
 
 # start the actual recording
 if ($pid == 0) {
  &log($spec_id,$title,"starting retrieval of $url with $get_prog (pid:$pid). executing $cmd \n",$logfile,$logging);
     exec $cmd;
 } # end if for child
 
 # parent process (this script).  we wait the duration of the program then kill the child
 elsif ($pid > 0) {
  
  if ($get_prog eq 'wget') {
   
   &wgetProgram($pid,$path,$spec_id,$title,$wget,$ext,$temp_filepath,$file_filepath,$final_filepath,$email_alerts,$sendmail,$contact_email,$contact_name,$duration_sec,$duration_min);
        
  } else {
   
   &mplayerProgram($pid,$path,$spec_id,$title,$khz,$mplayer,$sox,$normalize,$lame,$nice,$url,$ext,$temp_filepath,$file_filepath,
    $final_file,$final_filepath,$email_alerts,$sendmail,$contact_email,
    $contact_name,$duration_sec,$duration_min,$title,$author,$composer,$output_bitrate,
    $album,$year,$info_url,$genre,$date);
    
  } # end if /else for wget or mplayer 
  
  # tag our file
  if (-e $file_filepath) {
   &tagFile($title,$author,$composer,$year,$album,$file_filepath,$info_url,$genre);
  }
  
  # delete temp file and finish
  &cleanAndFinish($spec_id,$title,$file_filepath,$final_filepath,$get_prog,$temp_filepath,$email_alerts,$sendmail,$contact_email,$contact_name);
  
  # final check and completion
  if (-e $final_filepath) {
   # success!
   # if they want to auto-add stuff to itunes
   if ($add_to_itunes eq 'Yes') {
    my $return = &iTunes($final_filepath,$add_to_itunes_playlists,$add_to_itunes,$itunes_bookmarkable);
   }
   
   &success($spec_id,$final_filepath,$title,$info_url,$contact_email,$contact_name,$email_alerts,$action);
  } else {
   &fail($spec_id,"");
   my $message = "Final mp3 file wasn't found, problem moving from data directory to save location, $file_filepath -> $final_filepath";
   &log($spec_id,$title,$message,$logfile,$logging);
   if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
   &fail($spec_id,$temp_filepath,"$message");
   exit 1;
  }
 
 } # end if for parent
 
 # our fork() has failed, no child process is created!  this should obviously never happen
 elsif ($pid < 0) {
  my $message = "fork() failed... can not create child process to start mplayer";
  &log($spec_id,$title,$message,$logfile,$logging);
  if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
  &fail($spec_id,$temp_filepath,"$message");
  exit 1;
 }   
 
 my $ts_end = strftime( "%Y%m%d%H%M%S",localtime(time));
 &log($spec_id,$title,"ended recording of $title at $ts_end: $final_filepath",$logfile,$logging);
 

} else {

 print "no action passed\n";
 
}# end if/else for NO action (flags)

$dbh->disconnect();

######################### SUBROUTINES #########################

#################################################################################################
###################            CONFIGURATION-RELATED SUBROUTINES      #####################
#################################################################################################

sub getDbConfig {

 my $path_to_config = shift;
 
 my ($host,$database,$user,$mysqlpassword,$webmaster,$logging) = "";
 my @errors = ();
 
 # first check that the config exists
 if (not(-e $path_to_config)) {
  print "ERROR: Database configuration file not found\n";
  exit 1;
 }
 
 open(FILE,"$path_to_config");
 my @lines = ;
 close(FILE);
 
 foreach my $line (@lines) {
 
  if ($line =~ /^MYSQL_HOST:(.*?)$/) {
   $host = $1;
   $host = &trim($host);
  }
  elsif ($line =~ /^MYSQL_DATABASE:(.*?)$/) {
   $database = $1;
   $database = &trim($database);
  }
  elsif ($line =~ /^MYSQL_USER:(.*?)$/) {
   $user = $1;
   $user = &trim($user);
  }
  elsif ($line =~ /^MYSQL_PASSWORD:(.*?)$/) {
   $mysqlpassword = $1;
   $mysqlpassword = &trim($mysqlpassword);
  }
  elsif ($line =~ /^LOGGING:(.*?)$/) {
   $logging = $1;
   $logging = &trim($logging);
  }
  elsif ($line =~ /^WEBMASTER:(.*?)$/) {
   $webmaster = $1;
   $webmaster = &trim($webmaster);
  } else {
   #
  }

 } # end foreach thru lines of config file
 
 # make sure we have all our config info
 if (!$host) {
  push(@errors,"MYSQL host not found in config file");
 }
 if (!$database) {
  push(@errors,"MYSQL database not found in config file");
 }
 if (!$user) {
  push(@errors,"MYSQL user not found in config file");
 }
 if (!$mysqlpassword) {
  push(@errors,"MYSQL password not found in config file");
 }
 if (!$webmaster) {
  push(@errors,"Webmaster's email not found in config file");
 } else {
  # validate the email
  my $msg = &is_valid_email($webmaster);
  if (!$msg) {
   push(@errors,"Webmaster's email not formatted correctly: $msg");
  }
 }
 
 return ($host,$database,$user,$mysqlpassword,$webmaster,$logging,\@errors);
 
} # end sub getDbConfig

sub get_config {
 # get the streamshifter config stuff from the database

 my @errors = ();
 my $sql = "SELECT * FROM streamshifter_config LIMIT 1";
 my $sth = &db_query($sql);
 my ($self,$your_timezone,$id3_timestamp_format,$script_name,$path_to_cgibin,$path_to_library,$add_to_itunes,$auto_create_subdirs,$path_to_desktop,$path_to_rawdata,$mplayer,$sox,$normalize,$lame,$wget,$sendmail,$nice,$max_recording_hours) = "";
 while (my $ref = $sth->fetchrow_hashref) {
  $script_name=$ref->{'program_name'};
  $your_timezone=$ref->{'your_timezone'};
  $id3_timestamp_format=$ref->{'id3_timestamp_format'};
  $path_to_cgibin=$ref->{'path_to_cgibin'};
  $path_to_library=$ref->{'path_to_library'};
  $add_to_itunes=$ref->{'add_to_itunes'};
  $auto_create_subdirs=$ref->{'auto_create_subdirs'};
  $path_to_desktop=$ref->{'path_to_desktop'};
  $path_to_rawdata=$ref->{'path_to_rawdata'};
  $mplayer=$ref->{'path_to_mplayer'};
  $sox=$ref->{'path_to_sox'};
  $normalize=$ref->{'path_to_normalize'};
  $lame=$ref->{'path_to_lame'};
  $wget=$ref->{'path_to_wget'};
  $sendmail=$ref->{'path_to_sendmail'};
  $nice=$ref->{'path_to_nice'} . " ";
  $max_recording_hours=$ref->{'max_recording_hours'};
 }
 $sth->finish();
 $self = $path_to_cgibin . $script_name;
 
 # ERROR CHECKING
 
 # see if we have our directories and they exist
 if ((!$path_to_cgibin) || (not(-d $path_to_cgibin))) {
  push(@errors,"Path to cgi-bin not found, or path is not a directory: $path_to_cgibin");
 } 
 if ((!$self) || (not(-e $self))) {
  push(@errors,"Full path to program not found: $self");
 } 
 if ((!$path_to_library) || (not(-d $path_to_library))) {
  push(@errors,"Path to library not found, or path is not a directory: $path_to_library");
 } 
 if ((!$path_to_desktop) || (not(-d $path_to_desktop))) {
  push(@errors,"Path to desktop not found, or path is not a directory: $path_to_desktop");
 } 
 if ((!$path_to_rawdata) || (not(-d $path_to_rawdata))) {
  push(@errors,"Path to temp file dir not found, or path is not a directory: $path_to_rawdata");
 } 
 
 # if we don't have the program paths at this point, check the system
 if ((!$mplayer) || (not(-e $mplayer))) { $mplayer = stripSystemResponse(`/usr/bin/which mplayer`); }
 if ((!$sox) || (not(-e $sox))) { $sox = stripSystemResponse(`/usr/bin/which sox`); }
 if ((!$normalize) || (not(-e $normalize))) { $normalize = stripSystemResponse(`/usr/bin/which normalize`); }
 if ((!$lame) || (not(-e $lame))) { $lame = stripSystemResponse(`/usr/bin/which lame`); }
 if ((!$wget) || (not(-e $wget))) { $wget = stripSystemResponse(`/usr/bin/which wget`); }
 
 # if we don't have at least mplayer, wget and lame at this point, there's not much we can do
 if ((!$mplayer) || (not(-e $mplayer))) {
  push(@errors,"mplayer not found, or path is invalid: $mplayer");
 }
 if ((!$sox) || (not(-e $sox))) {
  push(@errors,"sox not found, or path is invalid: $sox");
 }
 if ((!$normalize) || (not(-e $normalize))) {
  push(@errors,"normalize not found, or path is invalid: $normalize");
 }
 if ( (!$lame) || (not(-e $lame)) ) {
  push(@errors,"lame not found, or path is invalid: $lame");
 }
 
 if ((!$wget) || (not(-e $wget))) {
  push(@errors,"wget not found, or path is invalid: $wget");
 }
 
 if ( ($add_to_itunes eq 'Yes') && (!checkForDarwin() )) {
  push(@errors,"Doesn't appear you're running darwin, so please select 'no' for 'add_to_itunes' in configuration.");
 }
 
 return ($self,$your_timezone,$id3_timestamp_format,$script_name,$path_to_cgibin,$path_to_library,$add_to_itunes,$auto_create_subdirs,$path_to_desktop,$path_to_rawdata,$mplayer,$sox,$normalize,$lame,$wget,$sendmail,$nice,$max_recording_hours,\@errors);

} # end sub get configuration

#################################################################################################
###################            SCHEDULE-RELATED SUBROUTINES        #####################
#################################################################################################

sub checkFlags {
 # checks command-line flags
 
 my $args_ref = shift;
 @ARGV = @$args_ref;
 
 my $arguments = @ARGV;
 my @valid_args = qw(-t -m -r -s -u);
 
 my ($test_id,$rec_id,$rec_url,$rec_minutes) = 0;
 my $hasvalid = 0;
 
 my $action = 'record';

 # check the values of the flags
 for (my $i=0; $i<$arguments; $i++) {
  # if the -t "test" flag
  if ($ARGV[$i] eq '-t') {
   $hasvalid = 1;
   if ($ARGV[$i+1]) {
    if ($ARGV[$i+1] =~ /^\d+$/) {
     $test_id = $ARGV[$i+1];
     $action = 'test';
    } else {
     push(@errors,"Value for -t (test) flag must be a positive integer - the ID of the record in the database");
    }
   } else {
    push(@errors,"The -t (test) flag needs a value - the ID of the program in the database");
   }
  } elsif ($ARGV[$i] eq '-r') {
   # if it's the "record" flag
   if ($ARGV[$i+1]) {
    $hasvalid = 1;
    if ($ARGV[$i+1] =~ /^\d+$/) {
     $rec_id = $ARGV[$i+1];
     $action = 'record';
    } else {
     push(@errors,"Value for -r (record) flag must be a positive integer - the ID of the record in the database");
    }
   } else {
    push(@errors,"The -r (record) flag needs a value - the ID of the program in the database");
   }
  } elsif ($ARGV[$i] eq '-u') {
   # it's the "URL" flag
   if (&validateURL($ARGV[$i+1])) {
    $rec_url = $ARGV[$i+1];
   } else {
    push(@errors,"Argument after -u (URL) flag must be a properly-formatted URL beginning with 'http' or 'mms'.  If you do not include this flag the default is the URL for the passed program ID in the database");
   }
  } elsif ($ARGV[$i] eq '-m') {
   # it's the "minutes" flag
   if ($ARGV[$i+1] =~ /^\d+$/) {
    $rec_minutes = $ARGV[$i+1];
   } else {
    push(@errors,"Value for -m (duration) flag must be a positive integer - number of minutes to record stream.  If you do not include this flag the default is 1");
   }
  } elsif ($ARGV[$i] eq '-s') {
   # it's the "check schedule" flag
   $hasvalid = 1;
   $action = 'check';
   #print "checking\n";
   #my $output = `/bin/echo \$PERL5LIB`;
   #print $output;
   
  } else {
   #
  }
 } # end for loop
 
 # if there's a url flag but no record or test flag, throw error
 if (($rec_url) && (!$action)) {
  push(@errors,"The -u (URL) flag must be used with one of -t (test) or -r (record) flags with the ID of the program in the database");
 }
 
 # if no flag found, error
 if ($hasvalid == 0) {
  push(@errors,"\nValid flags not found.  Valid flags:\n-s\tcheck schedule for recorded programs\n-r\trecord (next argument is the ID of program in the database)\n-t\ttest (next argument is the ID of program in the database)\n-m\tminutes to test (Optional. Next argument is number of minutes for testing. Used with -t. Default is 1.)\n-u\tURL of stream (Optional. Used with -r. Default is the URL for the program in the database.)\n");
 }
 
 # if test flag is set and valid but there's no minute flag, make minute 1
 if (($test_id) && (!$rec_minutes)) {
  $rec_minutes = 1;
 }
 
 my $id = 0;
 if ($test_id) { 
  $id = $test_id; 
 }
 if ($rec_id) { 
  $id = $rec_id;
 }
 
 # make sure the id exists in the db
 if ($id) {
  my $sql = "SELECT COUNT(*) FROM streamshifter_schedule WHERE id = '$id'";
  my $sth = &db_query($sql,$dbh);
  my $count = 0;
  while (my $ref = $sth->fetchrow_arrayref) {
   $count = $$ref[0]
  } # end while loop
  
  if ($count < 1) {
   push(@errors,"No corresponding program found for ID specified");
  }
 }
 
 return (\@errors,$id,$action,$rec_url,$rec_minutes);
 
} # end sub checkFlags

sub cleanTemp {
 # cleans any old temp files

 my ($max_file_age,$path_to_rawdata) = @_;
 opendir(DIR, $path_to_rawdata) || print "can't opendir $path_to_rawdata: $! to delete old files";
 my @files = readdir(DIR);
 closedir(DIR);
 shift(@files);
 shift(@files);
 foreach my $file (@files) {
  my $thisfile = $path_to_rawdata . $file;
  my $file_age = (-M $thisfile);
  if (($file_age > $max_file_age) && (-f $thisfile)) {
   unlink($thisfile);
   #print "deleting $thisfile because it is over $max_file_age days old\n";
  }
 }

} # sub cleanTemp

sub checkSchedule {
 # checks the database for programs scheduled to run
 
 my ($self,$max_file_age,$path_to_rawdata,$max_recording_hours,$hour,$minute,$day,$date,$path_to_library,$auto_create_subdirs,$add_to_itunes,$dbh) = @_;
 
 #print "checking schedule for $hour,$minute,$day,$date\n";

 &cleanTemp($max_file_age,$path_to_rawdata);
 
 # clean our program table/library of old or orhpaned program recordings
 &cleanPrograms($path_to_library,$auto_create_subdirs,$add_to_itunes,$script_name);
 
 # check for any zombie mplayer processes.  If so, kill them
 &checkZombie($max_recording_hours);
 
 my $programs_ref = &checkPrograms($hour,$minute,$day,$date);
 my @program_ids = @$programs_ref;
 
 # if array is empty, exit
 my $program_count = @program_ids;
 if ($program_count < 1) {
  exit 0;
 } else {
  # we have at least one program.  we SHOULD be be forking off record processes for each one but for SOME reason fork is causing me grief.  Instead we are forking off calls to this program with the record flag set, which does the trick
  foreach my $program (@program_ids) {
   my $program_pid = fork();
   # child process
   if ($program_pid == 0) {
    my $mycmd = "$self -r $program";
    &log($spec_id,"n/a","executing: $mycmd",$logfile,$logging);
    exec("$mycmd");
   } # end if for child 
  } # end loop thru programs
 } # end if/else for program count
 
 exit 0;
 
} # end sub checkSchedule

sub checkPrograms {
 # related to the checkSchedule sub

 my ($hour,$minute,$day,$date) = @_;
 # find any programs that may need to be recorded
 my @program_ids = ();
 my $sql0 = "SELECT * FROM streamshifter_schedule WHERE active = 'Yes' AND hour = '$hour' AND minute = '$minute'";
 #print "$sql0\n";
 my $sth0 = &db_query($sql0);
 while (my $ref0 = $sth0->fetchrow_hashref) {
  
  # check if the program is scheduled for TODAY
  my $record_program = &checkDay($ref0->{'repeat'},$day,$ref0->{'start_date'},$date);
  
  if ($record_program) {
   push(@program_ids,$ref0->{'id'});
   &log($ref0->{'id'},"n/a","found a program id to record: $ref0->{'id'}",$logfile,$logging);
  }
 }
 $sth0->finish();
 
 return (\@program_ids);

} # end sub checkPrograms

sub getDate {
 # get the date in the formats we need it.  We use mysql because it's waaaaaaay easier
 
 # we have to watch for timezones.  check timezone of schedule vs the tz we're in and adjust our date-getting accordingly
 my $input_timezone = shift;
 my ($adjust_min,$your_timezone) = &doTimezone($input_timezone);
 #print "adjust: $adjust_min\n";
 my $date_adj = "DATE_ADD";
 if ($adjust_min < 0) {
  $date_adj = "DATE_SUB";
  $adjust_min = ($adjust_min*-1);
 }
 
 my ($date,$hour,$minute,$day,$year,$ts,$ts2,$date_short);
 my $day_q = "
  SELECT 
  DATE_FORMAT(DATE_ADD(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE), INTERVAL 1 MINUTE),'%H%i') AS HHMM,
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%W') as DAY, 
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%H') as HOUR,
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%i') AS MINUTE, 
  DATE_FORMAT(${date_adj}(DATE_ADD(NOW(), INTERVAL $adjust_min MINUTE), INTERVAL 1 MINUTE),'%Y-%m-%d-%H-%i-%s') as TS, 
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%Y-%m-%d') as TODAY, 
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%Y') as YEAR, 
  DATE_FORMAT(${date_adj}(NOW(), INTERVAL $adjust_min MINUTE),'%c/%e/%y') AS DATE_SHORT
  ";
 #print "$day_q\n";
 my $sth_date = &db_query($day_q);
 while (my $ref = $sth_date->fetchrow_hashref) {
  $day = $ref->{'DAY'};
  ($hour,$minute) = $ref->{'HHMM'} =~ /^(\d\d)(\d\d)$/;
  #print "HOUR: $hour MINUTE: $minute\n";
  $date = $ref->{'TODAY'};
  $year = $ref->{'YEAR'};
  $ts = $ref->{'TS'};
  $ts2 = strftime( "%Y-%m-%d-%H-%M",gmtime(time)); # this is for the filename, kept in GMT so it's the same when music libraries in multiple timezones are synced
  $date_short = $ref->{'DATE_SHORT'};
 }
 $sth_date->finish();
 
 #my $timezone = strftime( "%Z",localtime(time));
 #&log('n/a','n/a',"home timezone: $your_timezone, current timezone: $timezone $date_adj $adjust_min",$logfile,$logging);
 return ($date,$hour,$minute,$day,$year,$ts,$ts2,$date_short,$your_timezone);

} # end sub getDate

sub doTimezone {
 # check the scheduled program's timezone vs. what the computer says the timezone we are IN is
 # allows the programs to record at the proper time when we are traveling
 
 my $sched_timezone = shift;
 my $timezone = strftime( "%Z",localtime(time));
 my %zones = (
  'AKST' => '9',
  'AST' => '4',
  'AKDT' => '8',
  'ADT' => '3',
  'CST' => '6',
  'CDT' => '5',
  'EST' => '5',
  'EDT' => '4',
  'HST' => '10',
  'MST' => '7',
  'MDT' => '6',
  'NST' => '3.5',
  'NDT' => '2.5',
  'PST' => '8',
  'PDT' => '7',
  'YST' => '8'
 );

 # hack on the schedule timezone to give it a proper timezone code
 $timezone =~ /([\w]{2})$/;
 my $suffix = $1;
 my $code = $sched_timezone;
 $code =~ s/[\w]$//;
 $code .= $suffix;
 
 my $utc_current = &round($zones{$timezone}*60);
 my $utc_sched = &round($zones{$code}*60);
 my $adjust_minute = ($utc_current-$utc_sched);
 
 return ($adjust_minute,$code);
 
} # end sub timezone

sub checkZombie {
 # checks for mplayer processes that have run for too long

 my $kill_after_hours = shift;
 my $max_time = $kill_after_hours*60*60;
 my @ps = `/bin/ps ax -o pid,etime,command | /usr/bin/grep mplayer`;
 chomp(@ps);
 
 foreach my $cur_proc (@ps) {
 
  # the fields of the processes (as names)
  my ($pid,$elapsed,$cmd) = $cur_proc =~ /^\s*(\d+)\s+([-:\d]+)\s+(.*)$/;
  $elapsed = &trim($elapsed);
  my ($h,$m,$s);
  if ($elapsed =~ /(\d\d):(\d\d):(\d\d)$/) {
   ($h,$m,$s) = ($1,$2,$3);
  } else {
   $elapsed =~ /(\d\d):(\d\d)$/;
   ($h,$m,$s) = (0,$1,$2);
  }

  # elapsed time in seconds
  my $eseconds = $h*60*60 + $m*60 + $s;

  # if we find an mplayer process greater than $kill_after_hours, we need to kill it
  if ($eseconds > $max_time) {
   # kill("15",$pid);
   my $kill = `/bin/kill -15 $pid`;
   &log("n/a","n/a","Sent a TERM signal to a zombie mplayer process pid $pid because it was running for $eseconds process: $cur_proc",$logfile,$logging);
  }
 
 } # end for thru processes

} # end sub checkzombie

sub cleanPrograms {
 # cleans records and info files of programs that are no longer in the filesystem
 my ($path_to_library,$auto_create_subdirs,$add_to_itunes,$script_name) = @_;
 
 # array to contain IDs of programs we need to delete
 my @orphaned = ();

 # first let's delete any records that don't have files
 my $sql = "SELECT id,path_to_file FROM streamshifter_programs";
 my $sth = &db_query($sql);
 while (my $ref = $sth->fetchrow_arrayref) {
  
  my $record = $$ref[0];
  my $program_path = $$ref[1];
  
  # check if the program is still on the filesystem
  if (not(-e $program_path)) {
   &log($record,"n/a","could not find file for recorded program, going to remove record: $record missing file: $program_path",$logfile,$logging);
   push(@orphaned,$record);
  }
 
 } # end while thru program recordings
 $sth->finish();
 
 # now let's query the schedule and see if any of our programs have more than their allotted number of episodes
 my $sql = "SELECT id,episodes_to_keep,title,add_to_itunes_playlists,itunes_bookmarkable FROM streamshifter_schedule";
 my $sth = &db_query($sql);
 while (my $ref = $sth->fetchrow_arrayref) {
  
  my $program_id = $$ref[0];
  my $episodes = $$ref[1];
  my $progtitle = $$ref[2];
  my $add_to_itunes_playlists = $$ref[3];
  my $itunes_bookmarkable = $$ref[4];
  
  # if episodes != \d then make the variable an impossibly high number so no files ever get expired
  if (not($episodes =~ /^\d/)) {
   $episodes = 100000;
  }
  
  # now look at each program and see how many episodes it has
  my $sql2 = "SELECT path_to_file FROM streamshifter_programs WHERE schedule_id = '$program_id'";
  my $count = 0;
  my @recorded = ();
  my $sth2 = &db_query($sql2);
  while (my $ref2 = $sth2->fetchrow_arrayref) {
   push(@recorded,$$ref[2]);
   #print "$$ref2[0]\n";
  }
  $count = @recorded;
  $sth2->finish();
  
  # if count is greater than number of allotted programs, sort by newest and delete the older ones over the limit
  if ($count > $episodes) {
   my $sql3 = "SELECT id,path_to_file,timestamp,path_to_file FROM streamshifter_programs WHERE schedule_id = '$program_id' ORDER BY timestamp DESC";
   my $sth3 = &db_query($sql3);
   my $pr_counter = 0;
   while (my $ref3 = $sth3->fetchrow_arrayref) {
    $pr_counter++;
    my $del_id = $$ref3[0];
    my $del_path = $$ref3[1];
    my $del_timestamp = $$ref3[2];
    my $fullpathtofile = $$ref3[3];
    if ($pr_counter > $episodes) {
     
     # if there are playlists specified, we need to delete the track from those playlists
     # ( to keep the lists clean )
     if ($add_to_itunes_playlists) {
      my $del_return = &playlistTrackDeleteCheck($program_id,$fullpathtofile,$add_to_itunes_playlists,$logfile,$logging);
     }
     
     push(@orphaned,$del_id);
     &log($program_id,"n/a","Program id $program_id has a max of $episodes episodes. Deleting older episode with id $del_id ($del_timestamp) and corresponding file",$logfile,$logging);
     if (unlink($del_path)) {
      &log($program_id,"n/a","deleted $del_path",$logfile,$logging);
     } else {
      &log($program_id,"n/a","Could not delete $del_path - it was probably deleted by other means",$logfile,$logging);
     }
    } # end if for counter > episodes
   } # end while thru this program
   $sth3->finish();   
   
  } # end if for count is greater than allowed episodes
  
  # if we have auto-created subdirs in the library (a subdirectory for each program), 
  # lets loop thru that directory in the filesystem and make sure that we have only our newest n
  # programs in there.  The reason I do this is sometimes I add files that were created elsewhere
  # (another campaign manager, wiretap pro, a podcast mp3, etc.)
  my $tempname = lc(&replaceSpace(&stripDateSpec($progtitle)));
  my $dir = $path_to_library . $tempname . '/';
  if (($auto_create_subdirs eq 'Yes') && (-d $dir)) {
   my @filelist = ();
   #open the directory
   opendir(DH, $dir) || die "Can't open directory $dir: $!\n"; 
   #sort them by creation date, newest file first, shove into array
   @filelist = sort { -M "$dir/$a" <=>  -M "$dir/$b" } grep { -f "$dir/$_" && /\.[MmOo][PpGg][3Gg]$/ } readdir DH;
   closedir(DH);
   
   # loop thru the files and mark any excess ones for deletion
   my $filecounter = 1;
   foreach my $file(@filelist) {
    my $fullpathtofile = $dir . $file;
    if ($filecounter > $episodes) {
     
     # if there are playlists specified, we need to delete the track from those playlists
     # ( to keep the lists clean )
     if ($add_to_itunes_playlists) {
      my $del_return = &playlistTrackDeleteCheck($program_id,$fullpathtofile,$add_to_itunes_playlists,$logfile,$logging);
     }
     
     # over the limit - DELETE and REMOVE FROM DB (doubtful it's in there, but just to be sure)
     if (unlink($fullpathtofile)) {
      &log($program_id,"n/a","Deleted $fullpathtofile because it was in the program dir and is number $filecounter when only $episodes are allowed",$logfile,$logging);
     } else {
      &log($program_id,"n/a","problem deleting $fullpathtofile",$logfile,$logging);
     }
     my $delquery = "DELETE FROM streamshifter_programs WHERE path_to_file = '$fullpathtofile'";
     my $del = &db_query($delquery);
     
    } else {
     # under the limit - leave alone, but...
     # check if the file has a corresponding entry in the database
     my $recordexists = 0;
     my $checkfile_q = "SELECT count(*) FROM streamshifter_programs WHERE path_to_file = '$fullpathtofile'";
     my $sth4 = &db_query($checkfile_q);
     while (my $ref4 = $sth4->fetchrow_arrayref) {
      $recordexists = $$ref4[0];
     }
     if (!$recordexists) {
      
      # first we need to check that this file isn't being handled by another process (lame)
      # this leads to issues with double-entry in the database and itunes playlists
      # if it is, we need to just exit
      #my @check_progs = qw(lame wget mplayer scp rsync bladeenc);
      my $exit = 0;
      &log($program_id,"n/a","checking if file $file is being handled by another process...",$logfile,$logging);
      my $pid = &checkPid2($file);
      if ($pid) { 
       $exit = 1;
      }
      # as a second check, see if streamshifter itself is operating on this program
      &log($program_id,"n/a","checking if file $file (program id $program_id) is being handled by $script_name...",$logfile,$logging);
      my $search_tmp = 'r ' . $program_id . ' ';
      my ($pid2,$filesize2) = &checkPid($script_name,$search_tmp);
      if ($pid2) {
       $exit = 1;
      }
      
      if ($exit) {
       &log($program_id,"n/a","file $file is being handled by another process, exiting",$logfile,$logging);
      } else {
       
       # we collect what info we can from the file and add a record in the db for it
       my $write_secs = (stat($fullpathtofile))[9];
       my $name_timestamp = strftime( "%A, %B %d %I:%M:%S %p %Z",localtime($write_secs));
       my $full_timestamp = strftime( "%Y%m%d%H%M%S",localtime($write_secs));
       my $record_timestamp = strftime( "%Y-%m-%d %H:%M:%S",localtime($write_secs));
       my $thisyear = strftime( "%Y",localtime($write_secs));
       my ($time,$bitrate,$filesize,$finaltitle) = &getDuration($fullpathtofile);
       my $name = "";
       if (!$finaltitle) {
        my $tmpname = &basename($fullpathtofile);
        $tmpname =~ s/_/ /g;
        $tmpname =~ s/\d\d\d\d-\d\d-\d\d//g;
        $tmpname = &trim($tmpname);
        $tmpname =~ s/\b(\w)/\u$1/g;
        $finaltitle = $tmpname . " - " . $name_timestamp;
        if ($fullpathtofile =~ /\.[Mm][Pp]3$/) {
         &id3v2tag($fullpathtofile,$finaltitle,$thisyear,"","","","","");
        }
       } else {
        if ($finaltitle =~ /Sunday|Monday|Tuesday|Wednesday|Thursday|Friday|Saturday/ig) {
         $name = $finaltitle;
        } else {
         $name = $finaltitle . " - " . $name_timestamp;
        }
       }
       $name = $dbh->quote($name);
       my $insertprog = "INSERT INTO streamshifter_programs SET
             schedule_id = '$program_id',
             title = $name,
             recording_date = '$record_timestamp',
             recording_length = '$time',
             bitrate = '$bitrate',
             filesize = '$filesize',
             path_to_file = '$fullpathtofile',
             info_url = '',
             timestamp = '$full_timestamp'
             ";
       #print "gonna insert $insertprog\n";
       my $insnew = &db_query($insertprog);
       if ($insnew) {
        &log($program_id,"n/a","inserted new program $name into recorded program listing",$logfile,$logging);
        if ($add_to_itunes eq 'Yes') {
         my $return = &iTunes($fullpathtofile,$add_to_itunes_playlists,$add_to_itunes,$itunes_bookmarkable);
        }
       }
      } # end if/else for file is being handled by another process.
     } # end NOT for record exists
    }
    $filecounter++;
   }
   
  } # end if for auto-create subdirs
  
  
 
 } # end while thru scheduled programs
 $sth->finish();
 
 # if we have any IDs, we need to delete
 if ($orphaned[0]) {
  my $ids = "'";
  $ids .= join("','",@orphaned);
  $ids .= "'";
  my $del = "DELETE FROM streamshifter_programs WHERE id IN($ids)";
  #print "Deleting program records with: $del\n";
  my $del_sth = &db_query($del);
  $del_sth->finish();
 }

} # end sub cleanPrograms

sub checkDay {
 # checks if the program is supposed to run TODAY
 
 my ($repeat_day,$day,$start_date,$date) = @_;
 
 my @weekend = qw(Saturday Sunday);
 my @weekdays = qw(Monday Tuesday Wednesday Thursday Friday);
 
 my $start_recording = 0;
 if ($repeat_day eq $day) {
  $start_recording = 1;
 } elsif (($repeat_day eq 'Once') && ($start_date eq $date)) {
  $start_recording = 1;
  #print "Starting one-time recording: $repeat_day $start_date eq $date\n";
 } elsif ($repeat_day eq 'Daily') {
  $start_recording = 1;
  #print "Starting daily recording: $repeat_day\n";
 } elsif ($repeat_day eq 'Weekdays') {
  my $found_weekday = 0;
  my $weekday_join = join(",",@weekdays);
  foreach my $weekday (@weekdays) {
   if ($day eq $weekday) {
    $found_weekday=1;
   }
  }
  if ($found_weekday) {
   $start_recording = 1;
   #print "Starting weekday recording: $repeat_day $day in $weekday_join\n";
  } else {
   $start_recording = 0;
   #print "NOT scheduled time: $repeat_day $day not in $weekday_join\n";
  }
 } elsif ($repeat_day eq 'Weekends') {
  my $found_weekend = 0;
  my $weekend_join = join(",",@weekend);
  foreach my $weekend (@weekend) {
   if ($day eq $weekend) {
    $found_weekend=1;
   }
  }
  if ($found_weekend) {
   $start_recording = 1;
   #print "Starting weekend recording: $repeat_day $day in $weekend_join\n";
  } else {
   $start_recording = 0;
   #print "NOT scheduled time: $repeat_day $day not not in $weekend_join\n";
  }
 } else {
  $start_recording = 0;
 }
 
 #print "start_recording: $start_recording\n";
 return $start_recording;

} # end sub checkDay

sub dateAnchors {
 # if replaces mysql-specific date anchors in strings with date info

 my ($string,$adjust,$interval,$interval_type) = @_;
 
 if (!$interval_type) {
  $interval_type = "HOUR";
 }
  
 # only do this if there are date specifiers in the string - we have to make sure they are url encodings (which can use similar specifiers, but use 2 characters instead of 1)
 if ( ($string =~ /%[abcDdefHhIijklMmprSsTUuVvWwXxYyZ]/) && (not($string =~ /%[abcdef][abcdef0-9]/)) ) {
  
  my $string2 = $string;
  my $timezone = strftime( "%Z",localtime(time));
  my $date_modifier = "DATE_ADD";
  if ($interval eq 'Past') {
   $date_modifier = "DATE_SUB";
  }
  if (!$adjust) {
   $adjust = 0;
  }
 
  while ($string =~ /(%[abcDdefHhIijklMmprSsTUuVvWwXxYy])/g) {
   
   my $dateq = "SELECT DATE_FORMAT($date_modifier(NOW(), INTERVAL $adjust $interval_type),'$1')";
   # this is kludgey, but if $adjust != 1 && $interval != 'Future' && $interval_type != MINUTE 
   #then we need to be sure that we adjust MYSQL's NOW() forward 1 minute since we start the process 1 minute before programs begine recording
   if (($adjust ne '1') && ($interval ne 'Future') && ($interval_type ne 'MINUTE')) {
    $dateq = "SELECT DATE_FORMAT($date_modifier(DATE_ADD(NOW(),INTERVAL 1 MINUTE), INTERVAL $adjust $interval_type),'$1')";
   }
   
   #print "$dateq\n";
   my $sthq = &db_query($dateq);
   my $replace = "";
   while (my $ref = $sthq->fetchrow_arrayref) {
    $replace = $$ref[0];
   }
   $sthq->finish();
   $string2 =~ s/$1/$replace/g;
  }
  
  # timezone is a little different.  %Z doesn't mean anything to mysql
  if ($string =~ /(%Z)/) {
   $string2 =~ s/$1/$timezone/g;
  }
  
  #print "found anchors and adjsuted string $string -> $string2\n";
  $string = $string2;
 }
 
 return($string);

} # end sub dateAnchors

#################################################################################################
###################    SUBROUTINES HAVING TO DO TESTING OF THE URL/STREAM   #####################
#################################################################################################

sub urlGetMethod {
 # creates the command we will use to save our program

 my ($url,$temp_filepath,$mplayer,$nice,$wget,$ext,$path_to_rawdata) = @_;
 my $cmd = "";
 my $get_prog = "";
 my $gonna_wget = 0;
 
 # actually determine the file type
 my ($filetype,$khz,$asfflag,$filetype_success,$failreason) = &determineFileType($url,$wget,$mplayer,$nice,$path_to_rawdata);
 
 # set mplayer cache settings if NOT an asf
 my $mplayer_cache = "";
 if (!$asfflag) {
  $mplayer_cache = "-cache 8192 -cache-min 4 ";
 }
 
 if ($filetype_success == 1) {
  if ($filetype eq "File") {
   # if it's not a stream, we need to wget it.
   $url =~ /\.([OoMm][GgPp][Gg3])/; 
   my $ext_tmp = lc($1);
   $temp_filepath =~ s/$ext/$ext_tmp/g;
   #print "EXT: $ext EXT_TMP: $ext_tmp TEMP_FILEPATH: $temp_filepath\n";;
   $cmd = "${nice}$wget '$url' -nv --output-document=$temp_filepath";
   $gonna_wget = 1;
   $get_prog = "wget";
   $ext = $ext_tmp;
  } elsif  ($filetype eq 'Stream') {
   #$cmd = "$mplayer '$url' -dumpstream -dumpfile $temp_filepath";
   $cmd = "${nice}$mplayer -really-quiet -bandwidth 5000000 ${mplayer_cache}-vc dummy -vo null -af volume=0,channels=2,format=s16le -srate 44100 -ao pcm:waveheader:file=$temp_filepath '$url'";
   if ($logging) { $cmd .= " >> $logfile 2>&1"; }
   $get_prog = "mplayer";
  } else {
   #$cmd = "$mplayer -playlist '$url' -dumpstream -dumpfile $temp_filepath";
   $cmd = "${nice}$mplayer -really-quiet -bandwidth 5000000 ${mplayer_cache}-vc dummy -vo null -af volume=0,channels=2,format=s16le -srate 44100 -ao pcm:waveheader:file=$temp_filepath -playlist '$url'";
   if ($logging) { $cmd .= " >> $logfile 2>&1"; }
   $get_prog = "mplayer";
  }
 }
 return ($filetype_success,$khz,$cmd,$get_prog,$temp_filepath,$ext,$failreason);
 
} # end sub urlGetMethod

sub determineFileType {
 # determines the type of url this is (file, stream, playlist).  related to urlGetMethod

 my ($url,$wget,$mplayer,$nice,$datadir)= @_;
 my $filetype = "";
 my $success = 0;
 my $sleep = 20; # DO NOT CHANGE THIS!  amount of time to get response from url - this sleep happens twice (once for wget test and once for mplayer test).  this entire process (sleep+sleep2) should take 46 seconds exactly. 
 my $sleep2 = 6; # DO NOT CHANGE THIS!  Since sleep*2+sleep2 = 46, mplayer should start trying to make the connection 14 seconds before the program's scheduled time
 my $failreason = "";
 my $failedflag = 0;
 my $khz = 0;
 my $asfflag = 0;
 
 # output log file and test sound file
 my $rand = &generate_random_string(12);
 my $log = $datadir . $rand . '.txt';
 my $test_sound = $datadir . $rand . '.wav';
 
 # only do this if the url begins with http
 if ($url =~ /^[Hh][Tt][Tt][Pp]:\/\//) {
 
  # usually it's a playlist, but we need to test
  $filetype = 'Playlist';
  
  # for the wget process
  my $wget_timeout = &round(($sleep-1)/2); # account for 2 tries
  
  my $cmd = "${nice}$wget --timeout=$wget_timeout -t 2 -o $log --output-document=/dev/null '$url'";
  &log("n/a","n/a","testing $url with $cmd",$logfile,$logging);
  my $wget_pid = fork();
    
  # child process
  if ($wget_pid == 0) {
   
   exec("$cmd");
   
  } elsif ($wget_pid > 0) {
   
   # wait, then grab the pid of the wget process and kill it
   sleep $sleep;
   my $killsuccess = &handlePid($wget,$url,$wget_pid,$log);
   if (!$killsuccess) {
    &log("n/a","n/a","Could not kill the $wget process! pid: $wget_pid",$logfile,$logging);
   }
   
  }
  
  # lets open our file and examine it
  open(FILE,"$log");
  my @lines = ;
  close(FILE);
  my $lengthfield = "";
  foreach my $line (@lines) {
   &log("n/a","n/a",$line,$logfile,$logging);
   if ($line =~ /^Resolving/) {
    if ($line =~ /failed/) {
     $failedflag = 1;
     $failreason .= "Failed resolving URL with this command:\n\n$cmd ";
    } 
   }
   elsif ($line =~ /[Bb]ad\sport/) {
    $failedflag = 1;
    $failreason .= "Bad port number.  Used this command:\n\n$cmd ";
   }
   elsif ($line =~ /404\s[Nn]ot\s[Ff]ound/) {
    $failedflag = 1;
    $failreason .= "Got 404 Not Found error - check URL for typos.  Used this command:\n\n$cmd ";
   }
   elsif ($line =~ /[Oo]peration\stimed\sout/) {
    $failedflag = 1;
    $failreason .= "Got 'Operation timed out' error with:\n\n$cmd ";
   }
   elsif ($line =~ /^Length/) {
    $line =~ /^Length: (.*)$/;
    $lengthfield = $1;
    if ($lengthfield =~ /x-ms-asf/) {
     $asfflag = 1;
    }
   } else {
    # nothing at this time
   }
  } # end foreach

  if ($failedflag == 0) {
   $success = 1;
   # what kid of url is it?
   if ($lengthfield =~ /unspecified/) {
    if ( ($lengthfield =~ /mpegurl/) || ($lengthfield =~ /scpls/) || ($lengthfield =~ /realaudio/) ) {
     $filetype="Playlist";
    } else {
     $filetype = 'Stream';
    }
   } else {
    # if it's over 150kb, let's treat it as a file
    $lengthfield =~ /^\s?(\d+)\s/;
    my $bytes = $1;
    if ($bytes >= 150000) {
     $filetype = 'File';
    } else {
     if ($asfflag==1) {
      #$filetype = 'Stream';
      $filetype = 'Stream';
     } else {
      $filetype = 'Playlist';
     }
    }
   }
  } else {
  
   $success = 0;
   
  }# end if for failedflag

 } else {
  
  # url doesn't begin with http, we assume it's a stream but we'll test next
  $filetype = 'Stream';
  $success = 1;
  &log("n/a","n/a","Appears to be a stream so we bypassed the wget check, sleeping $sleep until we test URL with mplayer.  URL: $url",$logfile,$logging);
  sleep $sleep;  
 }
 
 if ($success == 1) {
  &log("n/a","n/a","$url appears to be a $filetype",$logfile,$logging);
 }
 
 ### TEST THE STREAM IF IT'S NOT A FILE
 if (($filetype ne 'File') && ($success == 1)) {
 
  my $playlistmarker = "";
  if ($filetype eq 'Playlist') {
   $playlistmarker = ' -playlist';
  }
  
  # for the mplayer process
  #my $cmd = "$mplayer$playlistmarker '$url' -dumpstream -dumpfile $test_sound > $log 2>&1";
  my $cmd = "${nice}$mplayer -bandwidth 5000000 -cache 32 -cache-min 0 -nocache -vc dummy -vo null -af volume=0,channels=2 -ao pcm:waveheader:file=$test_sound $playlistmarker '$url' > $log 2>&1";
  &log("n/a","n/a","testing stream $url with $cmd",$logfile,$logging);
  my $mplayer_pid = fork();
    
  # child process
  if ($mplayer_pid == 0) {
  
   exec("$cmd");
   
  } elsif ($mplayer_pid > 0) {
   
   # wait, then grab the pid of the wget process and kill it
   sleep $sleep;
   my $killsuccess = &handlePid($mplayer,$url,$mplayer_pid,$test_sound);
   if (!$killsuccess) {
    &log("n/a","n/a","Could not kill the $mplayer process! pid: $mplayer_pid",$logfile,$logging);
   }
  }
  
  # lets open our logfile file and examine it
  open(FILE,"$log");
  my @lines = ;
  close(FILE);
  my $successflag = 0;
  foreach my $line (@lines) {
   if (($line =~ /^[Cc]onnected/) || ($line =~ /^ICY\sInfo/) || ($line =~ /^[Ss]tream\s[Nn]ot\s[Ss]eekable/) || ($line =~ /Audio\sstream\sfound/) || ($line =~ /Starting\splayback/)) {
    $successflag = 1;
   }
   if ($line =~ /^[Ee]rror/) {
    $line =~ s/\n//g;
    $failreason .= "${line}.\n";
   }
   if ($line =~ /^[Uu]nable/) {
    $line =~ s/\n//g;
    $failreason .= "${line}.\n";
   }
   if ($line =~ /^[Aa][Uu][Dd][Ii][Oo]: (\d+) Hz,/) {
    $khz = $1;
   }
   #print "LINE: $line";
  } # end foreach
  
  if ($successflag == 1) {
   # before assuming we have success, we need to see if there's an audiofile
   if (not(-e $test_sound)) {
    $success = 0;
    $failreason .= "\nmplayer could not create a test data file with this command:\n\n$cmd ";
   } else {
    $success = 1;
    #print "URL is valid and working...\n";
   }
  } else {
   $success = 0;
   $failreason .= "\nmplayer could not connect to server with this command:\n\n$cmd ";
  }
    
 } else {
  
  # just sleep so that we can stay on schedule
  &log("n/a","n/a","Appears to be a file so we bypassed the mplayer check, sleeping $sleep until we start getting the program.  URL: $url",$logfile,$logging);
  sleep $sleep;
  
 }# end if/else for NOT a file (stream testing)  
 
 # remove our temp file
 if (-e $log) {
  unlink($log);
 }
 # remove our temp file
 if (-e $test_sound) {
  unlink($test_sound);
 }

 if ($success == 1) {
  &log("n/a","n/a","sleeping $sleep2 more seconds until our program begins...",$logfile,$logging);
  sleep $sleep2;
 }


 return ($filetype,$khz,$asfflag,$success,$failreason);
 
} # end sub determineFileType

#################################################################################################
################### SUBROUTINES HAVING TO DO WITH RECORDING/SAVING/ENCODING #####################
#################################################################################################

sub getProgramInfo {
 # loads up our program's information (url, duration, title, etc)

 my ($spec_id,$action,$path_to_library,$path_to_desktop,$rec_url,$rec_minutes,$id3_timestamp_format,$dbh) = @_;
 my ($id,$adjust_info_url_interval,$path,$temp_file,$final_file,
     $temp_filepath,$file_filepath,$final_filepath,
     $ext,$user_id,$title,$author,$album,$composer,$genre,$url,$adjust_url,
     $adjust_url_interval,$info_url,$adjust_info_url,
     $duration_min,$duration_sec,$output_bitrate,$adjust_url,$add_to_itunes_playlists,$itunes_bookmarkable,
     $username,$contact_name,$contact_email,$email_alerts,$id3_timestamp);
 
 my $sql = "SELECT * FROM streamshifter_schedule WHERE id = '$spec_id' LIMIT 1";
 my $sth = &db_query($sql);
 while (my $ref = $sth->fetchrow_hashref) {

  # load the user associated with this program
  # if we don't hyave a userID, w eneed to die.
  if (!$ref->{'user_id'}) {
   my $message = "THERE IS NO USER ASSOCIATED WITH PROGRAM ID $spec_id!";
   &log($spec_id,$title,$message,$logfile,$logging);
   &fail($spec_id,"","$message");
   exit 1;
  }
  ($username,$contact_name,$contact_email,$email_alerts) = &loadUser($ref->{'user_id'});
  &log($spec_id,$title,"Loaded user: $username,$contact_name,$contact_email,$email_alerts",$logfile,$logging);
  
  # print timestamp
  my $ts_start = strftime( "%Y%m%d%H%M%S",localtime(time));
  &log($spec_id,$title,"starting recording of $ref->{'title'} at $ts_start",$logfile,$logging);
 
  # make our paths
  ($path,$temp_file,$final_file,$temp_filepath,$file_filepath,$final_filepath,$ext) = &getPaths($ref->{'title'},$action,$path_to_library,$path_to_desktop,$ref->{'save_to_library'});
  
  # get other data we need
  $id = $ref->{'id'};
  $user_id = $ref->{'user_id'};
  $title = &stripChars($ref->{'title'});
  $author = &stripChars($ref->{'author'});
  $album = &stripChars($ref->{'album_or_episode'});
  if (!$album) { $album = &stripDateSpec($title); }
  $composer = &stripChars($ref->{'composer_or_network'});
  $genre = &stripChars($ref->{'genre'});
  $url = &stripChars($ref->{'url'});
  if (length($rec_url)>0) { $url = $rec_url; }
  $adjust_url = $ref->{'adjust_url'};
  $adjust_url_interval = $ref->{'adjust_url_interval'};
  $adjust_info_url_interval = $ref->{'adjust_info_url_interval'};
  $info_url = &stripChars($ref->{'info_url'});
  $adjust_info_url = $ref->{'adjust_info_url'};
  $duration_min = $ref->{'duration_minutes'};
  if ($rec_minutes > 0) { $duration_min = $rec_minutes; }
  $duration_sec = ($duration_min*60); 
  $output_bitrate = $ref->{'output_bitrate'};
  $add_to_itunes_playlists = $ref->{'add_to_itunes_playlists'};
  $itunes_bookmarkable = $ref->{'itunes_bookmarkable'};
  $adjust_url = $ref->{'adjust_url'};
  
  # if the url has a date in it, replace the specifiers (MYSQL-compliant)
  if (length($id3_timestamp_format)>0) {
   $id3_timestamp = &dateAnchors($id3_timestamp_format,1,"Future","MINUTE");
   $title .= " - " . $id3_timestamp;
  } else {
   $title = &dateAnchors($title,1,"Future","MINUTE") ;
  }
  $author = &dateAnchors($author,1,"Future","MINUTE") ;
  $album = &dateAnchors($album,1,"Future","MINUTE");
  $composer = &dateAnchors($composer,1,"Future","MINUTE");
  $url = &dateAnchors($url,$adjust_url,$adjust_url_interval,"HOUR");
  $info_url = &dateAnchors($info_url,$adjust_info_url,$adjust_info_url_interval,"HOUR");
  
 } # end while thru db results
 $sth->finish();
 
 return ($path,$temp_file,$final_file,$temp_filepath,$file_filepath,$final_filepath,$ext,$user_id,$title,$author,$album,$composer,$genre,$url,$adjust_url,$adjust_url_interval,$info_url,$adjust_info_url,$duration_min,$duration_sec,$output_bitrate,$add_to_itunes_playlists,$itunes_bookmarkable,$username,$contact_name,$contact_email,$email_alerts);

} # end sub getProgramInfo

sub loadUser {
 # loads up the user associated with this program
 
 my $user_id = shift;
 
 my ($username,$contact_name,$contact_email,$email_alerts) = "";
 my $user_q = "SELECT * FROM streamshifter_users WHERE id = '$user_id' LIMIT 1";
 my $sth_user = &db_query($user_q);
 while (my $ref_user = $sth_user->fetchrow_hashref) {
  $username = $ref_user->{'username'};
  $contact_name = $ref_user->{'name'};
  $contact_email = $ref_user->{'email'};
  $email_alerts = $ref_user->{'email_alerts'};
 }
 $sth_user->finish();
 
 return ($username,$contact_name,$contact_email,$email_alerts);
 
} # end sub loadUser

sub getPaths {
 # creates the paths we will use for our various files

 my ($title,$action,$path_to_library,$path_to_desktop,$save_to_library) = @_;

 my $path = "";
 if ($save_to_library eq 'Yes') { 
  my $tempname = lc(&replaceSpace(&stripDateSpec($title)));
  my $dir = $path_to_library . $tempname . '/';
  my $checkdir = &checkLibDir($dir);
  if ($checkdir) {
   $path = $dir;
  } else {
   $path = $path_to_desktop;
  }
 } else {
  $path = $path_to_desktop;
 }# end if/else for save_to_library = Yes
 
 # if it's a test, we just set path to default_loc
 if ($action eq 'test') {
  $path = $path_to_desktop;
 }
 #print "will save file to $path\n";
 
 # create the name of our outputs file and make a full path to them
 # temporary file extension
 my $ext = 'wav';
 my $temp_file = &replaceSpace(&stripDateSpec($title)) . "_$ts2.$ext";
 my $final_file = &replaceSpace(&stripDateSpec($title)) . "_$ts2.mp3";
 my $temp_filepath = $path_to_rawdata . $temp_file;
 my $file_filepath = $path_to_rawdata . $final_file;
 my $final_filepath = $path . $final_file;
 
 return ($path,$temp_file,$final_file,$temp_filepath,$file_filepath,$final_filepath,$ext);
 
} # end sub getPaths

sub mplayerProgram {
 # this is the real meat - the mplayer rip and encode

 my ($pid,$path,$spec_id,$title,$khz,$mplayer,$sox,$normalize,$lame,$nice,$url,$ext,$temp_filepath,$file_filepath,
  $final_file,$final_filepath,$email_alerts,$sendmail,$contact_email,
  $contact_name,$duration_sec,$duration_min,$title,$author,$composer,$output_bitrate,
  $album,$year,$info_url,$genre,$date) = @_;
 
 # sleep x seconds to allow for a connection
 &log($spec_id,$title,"sleeping 14 seconds to allow for mplayer to connect",$logfile,$logging);
 sleep 14; # do not change this value
 
 # this is an mplayer stream
 &log($spec_id,$title,"sleeping $duration_sec seconds while program $title records",$logfile,$logging);

 # split our time into tenths, then loop thru our duration 10 times and be sure mplayer is running
 # this is so that if mplayer fails, our script doesn't keep running the entire duration of the program.
 my $time_split = ($duration_sec/10);
 my $filesize_prev = 0;
 for (my $i=0;$i<10;$i++) {
  
  if ($i==0){
   sleep $time_split;
  }
  my ($result,$filesize) = &checkPid($mplayer,$temp_filepath);
  if ($result) {
   #print "sub-sleeping $time_split\n";
   if ($i>0){
    my $message = "filesize at this point: $filesize bytes. Last time it was $filesize_prev";
    &log($spec_id,$title,$message,$logfile,$logging);
    sleep $time_split;
   }
  } else {
   my $message = "mplayer was found to not be running when it was supposed to be, cannot record $temp_filepath";
   &log($spec_id,$title,$message,$logfile,$logging);
   if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
   &fail($spec_id,$temp_filepath,$message);
   exit 1;
  }
  $filesize_prev = $filesize;
 } # end for loop thru time
 
 # kill mplayer
 #print "stopping mplayer process...\n";
 my $mpkill = &handlePid($mplayer,$url,$pid,$temp_filepath);
 if (!$mpkill) {
  &log($spec_id,$title,"Could not kill the $mplayer process! pid: $pid",$logfile,$logging);
 }
 
 # make sure we have a file at this point.  If not, we couldn't create the stream
 if (not(-e $temp_filepath)) {
  my $message = "No raw recording file was found: $temp_filepath";
  &log($spec_id,$title,$message,$logfile,$logging);
  if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
  &fail($spec_id,$temp_filepath,"$message");
  exit 1;
 }
 
 # encode as MP3
 &log($spec_id,$title,"converting $temp_filepath to $final_file",$logfile,$logging);
 # check encoder
 my $encode_string = "";
 
 # ID3 tags
 my $id3 = " --add-id3v2 --ignore-tag-errors --tt \"$title\" --ta \"$author $composer\" --tl \"$album\" --ty \"$year\" --tc \"$info_url\" --tg \"$genre\"";
 
 # do the encode
 # lame, normalize and sox are pretty intense, so let's check if it's already encoding another file and if so, wait
 my $process_check1 = &process_check($lame,'vbr-new',$spec_id,$temp_filepath,$action,$email_alerts,$sendmail,$contact_email,$contact_name);
 my $process_check2 = &process_check($sox,'resample',$spec_id,$temp_filepath,$action,$email_alerts,$sendmail,$contact_email,$contact_name);
 my $process_check3 = &process_check($normalize,'wav',$spec_id,$temp_filepath,$action,$email_alerts,$sendmail,$contact_email,$contact_name);
 
 if (($process_check1) && ($process_check2) && ($process_check3)) {
  # check to see if the file needs to be resampled
  ($temp_filepath,$khz) = &resample($temp_filepath,$khz,$sox,$nice,$spec_id,$title,$logfile,$logging);
  # normalize the wav file
  &normalize($temp_filepath,$normalize,$nice,$spec_id,$title,$logfile,$logging);
  $encode_string = "${nice}$lame -V0 -h -b $output_bitrate --quiet --vbr-new$id3 $temp_filepath $file_filepath"; 
 }   
 
 if ($logging) { $encode_string .= " >> $logfile 2>&1"; }
 &log($spec_id,$title,"encoding with command: $encode_string",$logfile,$logging);
 system("$encode_string");

} # end sub mplayerProgram

sub resample {
 # resamples the wav to something conventional
 
 my ($file,$khz,$sox,$nice,$spec_id,$title,$logfile,$logging) = @_;
 &log($spec_id,$title,"checking if we need to RESAMPLE with sox.  our khz is currently: $khz",$logfile,$logging);
 
 # check if the current khz is valid
 my @valid = qw(32000 44100 48000);
 my $hasvalidkhz = 0;
 foreach my $hz (@valid) {
  if ($hz==$khz) {
   $hasvalidkhz = 1;
  }
 }
 
 # change the khz if not valid
 if (!$hasvalidkhz) {
  
  my $newkhz;
  if ($khz < 32000) { $newkhz = 44100; }
  if (($khz > 32000) && ($khz < 44100)) { $newkhz = 44100; }
  if (($khz > 44100) && ($khz < 48000)) { $newkhz = 44100; }
  if ($khz > 48000)  { $newkhz = 44100; }
  
  # mv the file to a temp file
  $file =~ /^(.*?)\.wav$/;
  my $temp = $1 . '.tmp.wav';
  system("/bin/mv $file $temp");
  
  my $soxcmd = "${nice}$sox --ignore-length $temp -r $newkhz $file";
  if ($logging) { $soxcmd .= " >> $logfile 2>&1"; }
  &log($spec_id,$title,"resampling $file from $khz to $newkhz with $soxcmd",$logfile,$logging);
  system("$soxcmd");
  $khz = $newkhz;
  
  # delete our temp file
  if (-e $temp) {
   if(not(unlink($temp))) {
    &log($spec_id,$title,"could not delete $temp",$logfile,$logging);
   }
  }
  
 } # end if for validkhz
 
 return ($file,$khz);

} # end sub resample

sub normalize {
 # normalizes the wav to default normalize setting
 
 my ($file,$normalize,$nice,$spec_id,$title,$logfile,$logging) = @_;
 my $normcmd = "${nice}$normalize $file";
 if ($logging) { $normcmd .= " >> $logfile 2>&1"; }
 &log($spec_id,$title,"normalizing $file f with $normcmd",$logfile,$logging);
 system("$normcmd");

} # end sub normalize

sub wgetProgram {
 # if the url is a file, we use wget instead of mplayer
 
 my ($pid,$path,$spec_id,$title,$wget,$ext,$temp_filepath,
  $file_filepath,$final_filepath,$email_alerts,
  $sendmail,$contact_email,$contact_name,$duration_sec,
  $duration_min) = @_;
  
 # we just retrieve the file.  it's done when wget is no longer running. but we do want to kill wget after our $duration_sec if the file turns out to be a stream
 my $time_split = ($duration_sec/$duration_min);
 my $time_tracker = 0;
 for (my $i=0;$i<$duration_min;$i++) {
  my ($result,$filesize) = &checkPid($wget,$temp_filepath);
  if ($result) {
   &log($spec_id,$title,"sleeping $time_split filesize: $filesize",$logfile,$logging);
   sleep $time_split;
   $time_tracker = $time_tracker+$time_split;
   if ($time_tracker >= $duration_sec) {
    # kill wget
    &log($spec_id,$title,"stopping wget process: $result",$logfile,$logging);
    #kill("15",$result);
    my $kill = `/bin/kill -15 $result`;
   }
  } else {
   last;
  }
 } # end for loop thru time
 
 if (-e $temp_filepath) {
  my $message = "wget finished getting $temp_filepath";
  &log($spec_id,$title,$message,$logfile,$logging);
  # need to update our paths
  $file_filepath = $temp_filepath;
  $final_filepath = $path . &replaceSpace(&stripDateSpec($title)) . "_$ts2." . $ext;
 } else {
  my $message = "wget failed to save recording to: $temp_filepath";
  &log($spec_id,$title,$message,$logfile,$logging);
  if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
  &fail($spec_id,$temp_filepath,"$message");
  exit 1;
 }

} # end sub wgetProgram

sub tagFile {
 # tags our mp3 or ogg with meta data
 
 my ($title,$author,$composer,$year,$album,$file_filepath,$info_url,$genre,$use_mp3tag,$vorbiscomment,$my_encoder) = @_;
 
 # tag properly if it's an mp3 and we have the module
 my $tagtitle = "$title";
 my $tagauthor = "$author $composer";
 &id3v2tag($file_filepath,$tagtitle,$year,$tagauthor,$album,$info_url,$genre,"lame");
 
 return 1;

} # end sub tagFile

sub id3v2tag {
 # adds id3v2 tag to mp3s

 my ($file,$title,$year,$author,$album,$info_url,$genre,$encoder) = @_;

 # my $version = "$MP3::Tag::VERSION";
 eval {
     my $mp3 = MP3::Tag->new($file);
     $mp3->get_tags();
     $mp3->new_tag("ID3v2");
     $mp3->{ID3v2}->remove_tag();
     $mp3->{ID3v2}->add_frame("TALB", "$album");
     $mp3->{ID3v2}->add_frame("TIT2", "$title");
     $mp3->{ID3v2}->add_frame("TPE1", "$author");
     $mp3->{ID3v2}->add_frame("TCON", "$genre");
     $mp3->{ID3v2}->add_frame("TSSE", "$encoder");
     $mp3->{ID3v2}->add_frame("TYER", "$year");
	 #$mp3->{ID3v2}->add_frame('COMM', "$info_url", 'eng', "$info_url");
	 $mp3->{ID3v2}->frame_select_by_descr("COMM(eng,#0)[]","$info_url");
     #$mp3->{ID3v2}->add_frame("TIT3", "$info_url");
     #$mp3->{ID3v2}->add_frame("TRSN", "$info_url");
     #$mp3->{ID3v2}->add_frame('TXXX', "$info_url", 'eng', "$info_url");
     #$mp3->{ID3v2}->add_frame("WORS", "$info_url");
     #$mp3->{ID3v2}->add_frame("WXXX", "$info_url", 'eng', "$info_url");
     $mp3->{ID3v2}->write_tag;
     $mp3->close();
 }; warn $@ if $@;
return 1;

} # end sub id3v2tag

sub cleanAndFinish {
 # cleans up our recording programs
 
 my ($spec_id,$title,$file_filepath,$final_filepath,$get_prog,$temp_filepath,$email_alerts,$sendmail,$contact_email,$contact_name) = @_;
 if (-e $file_filepath) {
  if ($get_prog ne 'wget') {
   if (unlink($temp_filepath)) {
    &log($spec_id,$title,"removed $temp_filepath",$logfile,$logging);
   } else {
    my $message = "Could not remove $temp_filepath\n";
    if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
    &fail($spec_id,$temp_filepath,"$message");
    &log($spec_id,$title,$message,$logfile,$logging);
   }
  }
  system("/bin/mv $file_filepath $final_filepath");
 } else {
  if (-e $temp_filepath) {
   unlink($temp_filepath);
  } 
  my $message = "final output mp3 file ($file_filepath) was not encoded - problem with encoder or sox";
  &log($spec_id,$title,$message,$logfile,$logging);
  if ($email_alerts = 'Yes') { &emailAlert($sendmail,"Problem Recording \"$title\"",$message,"",$contact_email,$contact_name); }
  &fail($spec_id,$temp_filepath,"$message");
  exit 1;
 }

} # end sub cleanAndFinish

sub getDuration {
 # gets the duration of the file in question 

 my $filepath = shift;
 
 my $seconds = 0;
 my $bitrate = 0;
 my $finaltitle = "";
 
  # it's an mp3
  use MP3::Info;
  my $info = get_mp3info($filepath);
  my $tag = get_mp3tag($filepath);
  $seconds = $info->{SECS};
  $bitrate = $info->{BITRATE};
  $finaltitle = $tag->{TITLE};
  #print "TITLE IS: $info->{TITLE} $info->{ALBUM} $info->{ARTIST}\n";

 my $time = &convertSeconds($seconds);
 
 # get the file size
 my $fstemp = -s $filepath;
 my $fstemp2 = ($fstemp/1024);
 my $filesize = ($fstemp2/1024);
 $filesize = sprintf("%.1f", $filesize);
 
 return ($time,$bitrate,$filesize,$finaltitle);
 
} # end sub getduration

sub success {
 
 my ($id,$filepath,$title,$info_url,$htmldir,$contact_email,$contact_name,$email_alerts,$action) = @_;
 my $timestamp = strftime( "%Y%m%d%H%M%S",localtime(time));
 
 # get the duration
 my ($duration,$bitrate,$filesize,$finaltitle) = &getDuration($filepath);
 &log($id,$title,"recording length: $duration",$logfile,$logging);
 
 my $message = "New episode of $title successfully recorded.  Check it out!\n\nFile: $filepath\n\nDuration: $duration";
 &log($id,$title,$message,$logfile,$logging);
 if (($email_alerts = 'Yes') && ($action ne 'test')) { &emailAlert($sendmail,"New Episode: \"$title\"",$message,"",$contact_email,$contact_name); }
 $message = $dbh->quote($message);
 
 # update the scheduled program record with recent run info
 my $successq = "UPDATE streamshifter_schedule SET last_run = '$timestamp', last_run_result = 'Success', last_run_comments = $message WHERE id = '$id'";
 my $success = &db_query($successq);
 
 # add this row to the programs table
 $title = $dbh->quote($title);
 
 # first check to see that it's not already there
 my $count = 0;
 my $check_sql = "SELECT count(*) FROM streamshifter_programs WHERE path_to_file = '$filepath'";
 my $sthc = &db_query($check_sql);
 while (my $refc = $sthc->fetchrow_arrayref) {
  $count = $$refc[0];
 }
 if ($count == 0) {
  my $insertq = "INSERT into streamshifter_programs SET schedule_id = '$id', title = $title, recording_date = '$timestamp', info_url = '$info_url', recording_length = '$duration', bitrate = '$bitrate', filesize = '$filesize', path_to_file = '$filepath'";
  my $insert = &db_query($insertq);
 }

} # end sub success

sub fail {
 
 my ($id,$temp_filepath,$comments) = @_;
 my $timestamp = strftime( "%Y%m%d%H%M%S",localtime(time));
 $comments = $dbh->quote($comments);
 my $failq = "UPDATE streamshifter_schedule SET last_run = '$timestamp', last_run_result = 'Fail', last_run_comments = $comments WHERE id = '$id'";
 my $fails = &db_query($failq);
 if (-e $temp_filepath) {
  unlink($temp_filepath);
 }
 &log($id,"n/a","Rip failed, exiting: $comments",$logfile,$logging);
 
} # end sub fail

#################################################################################################
############  SUBROUTINES HAVING TO DO WITH CHECKING/KILLING RUNNING PROCESSES  #################
#################################################################################################

sub handlePid {
 # kills the PID that matches a certain criteria

 my ($grep1,$grep2,$possible_pid,$grep3) = @_;
 my @running = ();
 my $killflag = 0;
 my $pidgetter = "";
 if ($grep3) {
  $pidgetter = "/bin/ps ax -o pid,command | /usr/bin/grep '$grep1' | /usr/bin/grep '$grep2' | /usr/bin/grep '$grep3' | /usr/bin/grep -v 'grep' | /usr/bin/grep -v 'sh -c'";
 } else {
  $pidgetter = "/bin/ps ax -o pid,command | /usr/bin/grep '$grep1' | /usr/bin/grep '$grep2' | /usr/bin/grep -v 'grep' | /usr/bin/grep -v 'sh -c'";
 }
 
 &log("n/a","n/a","getting PID of running program with $pidgetter",$logfile,$logging);
 my @raw_pids = `$pidgetter`;
 #print "RAW\n";
 #print @raw_pids;
 chomp(@raw_pids);
 
 # if there are no raw_pids (shoudln't happen), throw in the possible pid from perl
 if (!$raw_pids[0]) {
  push(@raw_pids,$possible_pid);
 }
 
 foreach my $raw_pid (@raw_pids) {
  $raw_pid =~ s/^\s+//;
  $raw_pid =~ /^(\d+)\s/;
  $raw_pid = $1;
  my $pid = stripNonNumber($raw_pid);
  #print "PIDS: pid: $pid raw: $raw_pid\n";
  if ($pid) {
  my $exists = kill("0",$pid);
  if ($exists) {
   &log("n/a","n/a","found pid of running program: $pid ($grep1,$grep2,$grep3)",$logfile,$logging);
   push(@running,$pid);
  }
  }
 } # end foreach
 
 if ($running[0]) {
  # loop thru and kill the pids gently, then forcibly if necessary
  foreach my $pid (@running) {

   #my $killresult = kill("15",$pid);
   my $killresult = `/bin/kill -15 $pid`;
   
   &log("n/a","n/a","tried kill with: /bin/kill -15 $pid , got result: $killresult",$logfile,$logging);

   # check if it's still running'
   my ($killcheck1,$del) = &checkPid($grep1,$grep2);
   if ($killcheck1) {
   	# things are being killed properly, I just haven't had time to figure out why
   	# these incorrect messages are showing up in the log.  
    &log("n/a","n/a","Could not stop $pid with signal 15, trying with signal 9",$logfile,$logging);
	     my $killresult2 = `/bin/kill -9 $pid`;
	     &log("n/a","n/a","tried kill with: /bin/kill -9 $pid , got result: $killresult2",$logfile,$logging);
	     my ($killcheck2,$del) = &checkPid($grep1,$grep2);
	     if ($killcheck2) {
	      &log("n/a","n/a","still couldn't kill $pid, even with signal 9",$logfile,$logging);
	     } else {
	      $killflag = 1;
	      &log("n/a","n/a","$pid killed with signal 9",$logfile,$logging); 
	     }
   } else {
    $killflag = 1;
    &log("n/a","n/a","Stopped $pid with signal 15",$logfile,$logging);
   }


  } # end foreach thru pids
  if ($killflag == 0) {
   return 0;
  } else {
   return 1;  
  }
  
 } else {
  # no pids running, nothing to do.
  return 1;
 }
  
} # end handlePid

sub checkPid {
 # gets the PID of the process that matches a certain criteria

 my ($program,$grep) = @_;
 my @running = ();
 my $return_pid = "";
 
 # if the $grep var is a media file, let's get the filesize
 my $filesize = 0;
 if (($grep =~ /[MmOoWw][PpGgAa][3GgVv]$/) || ($grep =~ /mpfile$/)) {
  $filesize = (-s $grep);
 }
 my $pidgetter = "/bin/ps ax -o pid,command | /usr/bin/grep '$program' | /usr/bin/grep '$grep' | /usr/bin/grep -v 'grep' | /usr/bin/grep -v 'sh -c'";
 #print "checking for running program with $pidgetter\n";
 &log("n/a","n/a","checking for running program with $pidgetter",$logfile,$logging);
 my @raw_pids = `$pidgetter`;
 chomp(@raw_pids);
 
 foreach my $raw_pid (@raw_pids) {
  $raw_pid =~ s/^\s+//;
  $raw_pid =~ /^(\d+)\s/;
  $raw_pid = $1;
  my $pid = stripNonNumber($raw_pid);
  my $exists = kill("0",$pid);
  if ($exists) {
   my $message = "checking for running program $program (+$grep) I found one with pid ${pid}. ";
   &log("n/a","n/a",$message,$logfile,$logging);
   $return_pid = $pid
  }
 } # end foreach
 
 return ($return_pid,$filesize);

} # end sub checkPid

sub checkPid2 {
 # gets the PID of the process that matches a SINGLE criteria (different than checkPid since that one checks for two criteria)

 my ($text) = shift;
 my $return_pid = "";
 
 my $pidgetter = "/bin/ps ax -o pid,command | /usr/bin/grep '$text' | /usr/bin/grep -v 'grep' | /usr/bin/grep -v 'sh -c'";
 #print "checking for running program with $pidgetter\n";
 &log("n/a","n/a","checking for any running program with $pidgetter",$logfile,$logging);
 my @raw_pids = `$pidgetter`;
 chomp(@raw_pids);
 
 foreach my $raw_pid (@raw_pids) {
  $raw_pid =~ s/^\s+//;
  $raw_pid =~ /^(\d+)\s/;
  $raw_pid = $1;
  my $pid = stripNonNumber($raw_pid);
  my $exists = kill("0",$pid);
  if ($exists) {
   my $message = "checking for any program operating on $text. I found one with pid ${pid}. ";
   &log("n/a","n/a",$message,$logfile,$logging);
   $return_pid = $pid
  }
 } # end foreach
 
 return ($return_pid);

} # end sub checkPid2

sub process_check {
 # checks if a processor-intense process is already running, so that we can wait til its down before starting another

 my ($program,$grep,$id,$temp_filepath,$actions,$email_alerts,$sendmail,$contact_email,$contact_name) = @_;
 my $process_check_interval = 20; # the number of seconds to wait in between process checks (to see if our file is in the midst of uploading)
 my $process_check_loop_timeout = 100; # the number of times we want to check if a file is being uploaded - this prevents an infinite loop if a process is hung
 my ($grep_output,$filesize) = &checkPid($program,$grep);
 my $loop_check = 0;
 my $fail = 0;
 while ($grep_output ne "") {
  $loop_check++;
  sleep $process_check_interval;
  ($grep_output,$filesize) = &checkPid($program,$grep);
  &log($id,"n/a","$program is already running waiting $process_check_interval secs ($loop_check) output: $grep_output",$logfile,$logging);
  
  if ($loop_check >= $process_check_loop_timeout) {
   # this will hopefully prevent an infinite loop in the event a process is hung
   my $message = "process checking timeout reached - aborting.  Process check interval: $process_check_interval looped: $loop_check output: $grep_output\n";
   &log($id,"n/a",$message,$logfile,$logging);
   $fail = 1
   &fail($id,$temp_filepath,$message);
   if (($email_alerts = 'Yes') && ($action ne 'test')) { &emailAlert($sendmail,"ERROR",$message,"",$contact_email,$contact_name); }
   exit 1;
  }
 }
 return 1;

} # end sub process_check

#################################################################################################
###################                ITUNES-SPECIFIC SUBROUTINES           #####################
#################################################################################################

sub iTunes {
 # handles vairous itunes tasks that will only work on a mac os x
  
 my ($file,$add_to_itunes_playlists,$add_to_itunes,$itunes_bookmarkable) = @_;

 # check that we're on Darwin and only do this stuff if we are
 # and check that itunes is running
 my $darwin = checkForDarwin();
 my $itunes_running = checkForiTunes();
 if ( ($darwin) && ($add_to_itunes eq 'Yes') && ($itunes_running) ) {

  # if the files exists, add to itunes library
  my $finder;
  if (-e $file) {
   &log("n/a","n/a","adding $file to iTunes Library",$logfile,$logging);
   $finder = addToiTunes($file);
  }
  
  # add it to a itunes playlist(s)
  if ( ($add_to_itunes_playlists) && ($finder) ) {
   
   # split the string into an array on commas (can contain multiple playlists)
   my @playlists = split(/,/,$add_to_itunes_playlists);
   
   # loop thru the playlists...
   foreach my $itunes_playlist (@playlists) {
   
    $itunes_playlist = trim(stripNonAlph($itunes_playlist));
    # check if the pl exists
    my $pl_check = checkPlaylistExists($itunes_playlist);
    if ( (not($pl_check =~/^\d/)) || (not($pl_check >= 1 )) ) {
     # create the playlist
     $pl_check = createPlaylist($itunes_playlist);
    } # end if for playlist does not exist
    
    if (($pl_check =~/^\d/) && ($pl_check >=1 )) {
     # check to make sure playlist(s) exist
     &log("n/a","n/a","adding file $file ($finder) to $itunes_playlist playlist",$logfile,$logging);
     my $return = addToPlaylist($finder,$itunes_playlist);
    } # end if for playlist exists
   
   } # end loop thru itunes playlists
  } # end if for add to playlists
  
  # set the track to bookmarkable
  if (($finder) && ($itunes_bookmarkable eq 'Yes')) {
   &log("n/a","n/a","setting file $file ($finder) to bookmarkable/no shuffle",$logfile,$logging);
   &setBookMarkable($finder);
  }
 
 } # end if for is a mac and wants stuff added to lib
 
 return 1;

} # end sub iTunes

sub checkForDarwin {
 # checks that we're running darwin
 
 my $darwin = `/usr/bin/uname -s`;
 $darwin = trim(stripNonAlph(lc($darwin)));
 if ($darwin =~ /^darwin/) {
  return 1;
 } else {
  return 0;
 }

} # end sub for check for darwin

sub addToiTunes {
 # adds the file to the itunes library
 
 my $file = shift;
 my $ref = `/usr/bin/osascript -e 'tell application "iTunes" to add POSIX file "$file"'`;
 $ref = &stripNonAlph($ref);
 sleep 60; # sleep so we can allow itunes to process it.  takes a while for larger files
 return ($ref);
 
} # end sub addToiTunes

sub addToPlaylist {
 # adds the file to a particular playlist
 
 my ($finder,$playlist) = @_;
 my $ref=`/usr/bin/osascript <fetchrow_arrayref) {
   $del_title = $$name_ref[0];
  }
  # split the string into an array on commas (can contain multiple playlists)
  my @playlists = split(/,/,$add_to_itunes_playlists);
  
  # loop thru the playlists...
  foreach my $itunes_playlist (@playlists) {
  
   $itunes_playlist = trim(stripNonAlph($itunes_playlist));
   # check if the pl exists
   my $pl_check = checkPlaylistExists($itunes_playlist);
   if (($pl_check =~/^\d/) && ($pl_check >=1 )) {
    my $ret = &deletePlaylistTrack($itunes_playlist,$del_title);
    &log($program_id,"n/a","deleted $del_title from playlist $itunes_playlist",$logfile,$logging);
   } # end if for playlist exists
   
  } # end foreach thru playlists
 } # end if for $add_to_itunes_playlists 
     
} # end sub playlistTrackDeleteCheck

sub deletePlaylistTrack {
 # delete track from playlist
 # part of "cleanup"
 
 my ($playlist,$songname) = @_;
 my $delete = "";
 if (-e "/usr/bin/osascript") {
  $delete=`/usr/bin/osascript <prepare("$query"); 
 #print $query;
 $sth->execute;
 #print $dbh->err; 
 my $err = $dbh->err;  
 my $errstr = $dbh->errstr;
 if ((length($errstr)>0)) {
  &emailAlert($sendmail,$err,$errstr,$query,$webmaster);
 }

 return $sth;
 
} # end sub db_query

sub stripChars {

 my($text) = @_;
 
 $text =~ s/\n/ /g; # strip carraige returns
 $text =~ s/\t/ /g; # strip tabs
 $text =~ s/\a/ /g; # strip carraige returns
 $text =~ s/"/'/g; # strip quotes and replace with single quotes
 $text =~ s/\s+/ /g; # strip repeating spaces and replace with one
 return ($text);

} # end sub stripchars

sub stripSystemResponse {

 my($text) = @_;
 if ($text) {
  $text =~ s/([^A-Za-z0-9\/])//g; 
 }
 return ($text);

} # end sub stripSystemResponse

sub stripNonNumber {

 my($text) = shift;
 $text =~ s/([^0-9])//g;
 return ($text);

} # end sub strip non number

sub stripNonAlph {

 my($text) = shift;
 $text =~ s/([^0-9a-zA-z ])//g;
 return ($text);

} # end sub strip non alph

sub stripDateSpec {

 my $text = shift;
 if ($text =~ /%/) {
  $text =~ m/^(.*?)%/;
  $text = $1;
 } 
 $text =~ s/^\s+//; # strip leading and trailing spaces
 $text =~ s/\s+$//;
 $text =~ s/[^\w]$//; # strip out any trailing characters that aren't letter or numbers
 $text =~ s/\s+$//; # make double sure the last char isn't a space
 return $text;

} # end sub stripdatespec

sub replaceSpace {

 my($text) = shift;
 #$text = s/(%[abcDdefHhIijklMmprSsTUuVvWwXxYy])//g; # strip out mysql date specifiers
 $text =~ s/([^\w+\s+])//g;
 $text =~ s/^\s+//;
 $text =~ s/\s+$//;
 $text =~ s/([\s+])/_/g;
 return ($text);

} # end sub replacespace

sub trim {

 my $text = shift;
 $text =~ s/^\s+//;
 $text =~ s/\s+$//;
 return $text;
}

sub round {
    my($number) = shift;
    return int($number + .5);
}

sub log {

 my ($id,$title,$message,$logfile,$logging) = @_;
 my $timestamp = strftime( "%Y-%m-%d %H:%M:%S",localtime(time));
 if (($message) && ($logfile) && ($logging)) {
  $message =~ s/\n/ /g;
  $message =~ s/\s+/ /g;
  open(FILE,">>$logfile") || die "could not open $logfile for writing $!";
  print FILE "$timestamp\t$id\t$title\t$message\n";  
  close(FILE);
  return 1;
 } else {
  return 0;
 }
 
} #end sub log

sub emailAlert {

 my ($sendmail,$alert,$message,$sql,$contact_email,$contact_name) = @_;
 my $timestamp = strftime( "%Y%m%d%H%M%S",localtime(time));
 my $subject = "StreamShifter Alert - $alert\n";
 my $contact = $contact_email;
 if ($contact_name) { $contact = $contact_name; }
 my $email_content = "$contact,\n\n$alert: $message\n$sql\nSincerely,\n\nStreamShifter\n\nTimestamp: $timestamp\n\n";
 if (&Mail($sendmail,$contact_email,$subject,$email_content)) {
  #print "alert below.  Emailed $contact_email\n\n$alert: $message\n";
 } else {
  #print "tried to email $contact_email about alert but was unsuccessful\n";
 }

} # end sub emailAlert

sub Mail {

my ($Sendmail,$recipient,$subject,$email_content,$from) = @_;
if (!$from) {
 $from = $recipient;
}
open(MAIL, "| $Sendmail -f $from -t") || die "Couldn't open a pipe to Sendmail: [$Sendmail].  Please notify $from of this error.";
print MAIL "From: $from\n";
print MAIL "To: $recipient\n";
print MAIL "Subject: $subject\n\n";
print MAIL <<"EOF";
$email_content\n
EOF
close(MAIL);

}

# for getting the basename of a file without using a module
sub basename {
 my $file = shift;
 $file =~ s!^(?:.*/)?(.+?)(?:\.[^.]*)?$!$1!;
 return $file;
} # end sub basename

################# NOT USED AT THIS TIME ##############
sub adjustTime {

 my ($hour,$minute) = @_;
 
 # remove leading zeros from minute and hour so we can do math on them
 if ($minute =~ /^0([0-9])$/) {
  $minute = $1;
 }
 if ($hour =~ /^0([0-9])$/) {
  $hour = $1;
 }
 # if the minute is 0, subtract 1 from the minute.  if the minute is 0 and the hour is 0, the hour becomes 23.
 if ($minute == 0) {
  $minute = 59;
  if ($hour == 0) {
   $hour = 23;
  } else {
   $hour = ($hour-1);
  }
 } else {
  $minute = ($minute-1);
 }
 # put the leading zeros back on single digits
 if ($minute =~ /^([0-9]{1})$/) {
  $minute = '0' . $1;
 }
 if ($hour =~ /^([0-9]{1})$/) {
  $hour = '0' . $1;
 }
 return($hour,$minute); 
 
} # end sub adjustTime

# EOF

Finally, it needs a cronjob to run it. Edit the path and add to your crontab:
* * * * * /home/user/bin/streamshifter.cgi -s
Once you have it set up, you admin the programs in the MySQL database (see the test data in the "streamshifter_schedule" table. There are also a few things you can do on the command line with it. run:
./streamshifter.cgi -h
To see the options.