Categories
Computer science programming

Whisper and WhisperX for podcast transcription

I just realized that I hadn’t posted this. Several months ago, I read about whisper.cpp and started playing with it. To quote from their docs, whisper.cpp is a

High-performance inference of OpenAI’s Whisper automatic speech recognition (ASR) model:

  • Plain C/C++ implementation without dependencies
  • Apple Silicon first-class citizen – optimized via ARM NEON, Accelerate framework, Metal and Core ML
https://github.com/ggerganov/whisper.cpp

In other words, a fast and free speech transcription app that runs on your laptop. Damn!

In fact, it’s so efficiently written that you can transcribe on your iOS phone. Or browser. Haven’t tried those yet.

Anyway, that gave me an idea: a couple of friends of mine run a podcast called TGN. They’ve been at it for a few years, and have around 250 episodes of an hour each. Could I use whisper.cpp to produce a complete set of episode transcripts? If I did, would that be useful?

(Insert a few months of side project hacking, nights and weekends.)

It works, pretty well. For podcasts, however, you end up with a wall of text because Whisper doesn’t do what’s called ‘speaker diarization,’ that is, identifying one voice or another. It’s on their roadmap, though.

I was sharing the progress on the TGN slack when an employee of the company OctoML DM’d me. They have a WhisperX image that does diarization, and he offered to help me use it for the project.

(More nights and weekends. Me finding bugs for OctoML. Adding a second podcast. Getting help from a couple of friends, including the a-ha mkdocs idea from David.)

Voila! May I present:

The key useful bits include

  • Full text searching
  • Local mirrored copies of the podcast MP3, raw transcript text and (in progress attempts) to mirror the episode web page.
  • Using mkdocs to build static websites that look OK, and can render Markdown into HTML.

Lots of features yet to build but it’s been a really fun side project. Source code is all on GitHub, right now I’m in the prefect branch, trying out a workflow rewrite using Prefect.

Categories
Aphorisms

For a friend who just got laid off

41. If you do everything for one reason, then all you have done will be become meaningless when the reason does.

James Richardson, Vectors: Aphorisms and Ten-Second Essays

Categories
Aphorisms

Richardson, au courant

26.
The tyrant’s self-esteem is just fine, thank you. It’s you he
doesn’t care much for. And yes, he recognizes that he doesn’t
feel what you feel. Which is a good thing, since your feeling is
so weak that it makes him need to beat you up.

James Richardson, Vectors 3.0: Even More Aphorisms and Ten-Second Essays

Categories
programming

Internet manners – download if newer

If you’re a developer, sooner or later you’ll need to poll a website or URL, looking for information. There are good and bad uses of this, (looking at you, Ticketmaster and scalpers) and I want to be good. (This was inspired by reading Rachel’s post about her rate limiter.)

As part of my podcast speech-to-text project I am downloading a chunk of XML from their RSS feeds. These are commercial and can handle lots of traffic, but just the same why not be smart and polite about it? My goal is to have a cron job polling for new episodes, and I don’t want to cost them excess hosting and traffic fees.

The HTTP spec includes a header called Last-Modified, so your first thought might be:

  • Use the HTTP ‘head’ verb to only get metadata
  • Check the Last-Modified header to see if its newer

This will not work. ‘HEAD’ doesn’t include Last-Modified. Instead, you need the ‘ETag’ header! It’s roughly (see that link for details) a file hash, so if the RSS updates the ETag should as well. Simply save the ETag to disk and compare the version returned in the ‘HEAD’ metadata. Fast, polite, simple. Here’s the Python version from the project:

def podcast_updated(podcast: Podcast) -> bool:
    # Based on our saved last-updated time, are there new episodes? If not, don't
    # hammer their server. Internet manners. Method - call HEAD instead of GET
    # Note that HEAD doesn't include a timestamp, but does include the cache ETag, so
    # we simply snapshot the etag to disk and see if it differs.
    filename = podcast.name + '-timestamp.txt'
    try:
        r = requests.head(podcast.rss_url)
        url_etag = r.headers['ETag']
        file_etag = open(filename, 'r').read()

        if file_etag == url_etag:
            log.info(f'No new episodes found in podcast {podcast.name}')
            return False
    except FileNotFoundError:
        log.warning(f'File {filename} not found, creating.')

    open(filename, 'w').write(url_etag)
    return True

It adds a local file per URL, which is a bit messy and I need to rework the code a bit, but it’s working and runs quite fast.

Another idea would be to try the requests-cache module, which does this plus local caching of content. I’ve read their docs but not tried it yet.

P.S. Yes, I also shared this on StackOverflow. 😉

Categories
Uncategorized

Goddamn WordPress

So I pay for WordPress to host this blog. Money well spent at 48/year. I also run a private blog. The WordPress iOS app as well as their new Jetpack app both refuse to log into the self hosted instance.

Why? I’m guessing fuck you, pay me. The BlogPost app kind of works but I need to work on its image uploading.

Had to rant. Guessing there are lots of others in this predicament.

Categories
Uncategorized

Mastodon

Well, Twitter blocked my client app Tweetbot and I’ve finally setup Mastodon. At a friends’ recommendation I’ve created @pfhubbard (Mastodon) and am finding the people that I used to read/follow.

What a goddamn catastrophe.

Categories
Music

Wonderful performance of JS Bach

I’m a fan of JS Bach in general and of his pipe organ work in particular. My mom was sometimes a church organist, so I’ve memories of reading while she practiced, though always with synths and not a full organ. I did get to hear the instrument at Cordiner Hall many times while working there, complete with hunchbacking down a five foot tall tunnel underneath the seats while it was played… anyway.

This is an excellent performance, a bit different than my favorite by Marie-Claire Alain, but its well recorded and definitely wonderful. Good headphones recommended, otherwise the pedal tones don’t come through as they should. That final chord resolution, singing through the hall, just goosebump inducing. Enjoy.

Categories
recommendations

Clothing for tall men

This is a topic for a small percentage of people, but the Internet is vast so I’ll post it for others who’ll find it useful.

By way of context, I am 6’10” (208cm) and 260lb (118kg). I’m looking for daily clothes for office work mostly, and exercise gear as well with a strong focus on longevity and environmental impact. I do have a couple suits and a tuxedo but those mostly sit idle so they’ll be discussed as a side topic. Let’s go!

My go-to, trusted brands for daily wear

  • Eddie Bauer – pay attention to fit descriptions (classic versus relaxed) but many of my clothes are EB. Their modular winter coat was great for Midwestern winters.
  • Land’s End – similar notes here. For a while, they were owned by Sears and you could try stuff on, but even as web-orders they are solidly reliable.
  • LL Bean – ditto, though I think a slightly smaller selection of tall sizes.
  • String King – this is an awesome find. A lacrosse brand from Los Angeles, but they make custom t-shirts, hoodies and quarter-zips. You tell the site your measurements and they ship it to you, complete with your name on the tag. I’ve got on t-shirt from them, in the poly blend, and its holding up well and seems an excellent value, so I plan to shop here again.
  • Levi’s – the 541 athletic fit is my daily. I take each pair to a local tailor to add two more belt loops on the back, since they are made with only a single rear loop, and then I wear the hell out of them. 2% Lycra/elastane. Wrangler and Carhartt also make tall sizes and I’ve a pair of each, but save them for yardwork, mud and camp outs as I find them less comfortable than the Levi’s.
  • J. Crew – I’m only a bit of their clothing. Sometimes excellent.
  • Darn Tough – their merino-blend socks are expensive at ~20/pair but they work superbly, last a long time and are fully guaranteed for life. Despite having size 15 feet, I prefer their XL size to the fit of the XXL.
  • Ex Officio – their give and go travel brief is expensive at $26 but last. I have a couple of weeks’ worth and they’ve held out well.

New discoveries I’ve not yet purchased

  • American Tall – sizes up to 7’1”. I’ve a nephew who likes their clothing.
  • Navas Labs – recommended by family.
  • Tommy Bahama – I’m not really the Parrothead type that I associate with the brand but their tall size listing actually looks promising.
  • The Gap – I’ve few from them but they’re well regarded
  • Norrøna. Similar to Patagonia in that they don’t sell tall sizes, but my 6’2” sibling tells me that their torsos and sleeves are long enough. Premium prices and reputation. They also have a mountain bike lineup with longer torsos and sleeves.
  • A relative (6’2”) strongly recommends the European brand Devold. Their stuff is well made, proportioned for tall folks despite not being labeled as such and is durable and well made. A quick search found this importer, so I may give them a try.

Exercise clothing

  • AeroTech Designs makes mountain biking jerseys and shorts. I’ve their long-sleeve merino jersey and it’s great.
  • The tech shirts from Eddie Bauer are my go-to here. Since learning that microfibers from polyester are a significant pollutant, I’m searching for merino replacements but haven’t found one yet. Contact me if you can help!
  • Patagonia does not make tall sizes but they seem to make their cycling jerseys long in the torso since cycling requires a bent-forward position. I have two pairs of their Dirt Craft shorts and the 3/4 sleeve merino jersey, and they are worth every penny and fit pretty well.
  • Kitsbow makes high-end outdoor and cycling clothing to order, in America. It’s beautiful stuff and quite expensive. They now have a few tall items! I’ve one of their hot-weather jerseys (Superflow Cooling Tee) (took a chance on a non-tall XL size) and it’s damned good at wicking. I admire them as a company and, if you read up on them, I think you might also. They recently introduced a used-gear store called Experienced Gear.

Information sources

I skim via RSS as explained here, and two sites I’ve come to trust are Dappered and Put This On. String King and Taylor Stitch, for example, were both via Dappered.

Formal wear

Back when I was doing more business travel, a friend recommended Sam’s Tailor, on the Kowloon side of Hong Kong. I’ve been there twice, and gotten a suit, two blazers, several dress shirts and some dress pants. It’s been years since I was there though. My tuxedo was a random purchase on a trip to Singapore, and the price and experience were meh so I’d recommend Sam’s there also.

Note that Sam’s does women’s clothing too – I took one of my wife’s favorite shirts along and they made a couple copies in silk for her.

I hope that this list helps someone else out. Do contact me or leave a comment, please.

Categories
Uncategorized

Today I learned

WordPress mobile will throw errors about missing XML-RPC functionality if your PHP installation is missing the ‘dom’ library.

Took a while, that did. 

apt install php-dom
Categories
recommendations

DIY MagSafe stand

So there are fifty to one hundred dollar stands that hold a MagSafe puck for your phone on your desk. Save your money – this is a 20 dollar stand with a MagSafe attached using two 3M Command Strips.

Less classy

I got the stand from Amazon(non referral link). Tablet Stand Adjustable, Lamicall. The strips are “Command Poster Hanging Strips Value-Pack, Small, White, 48-Pairs (17024-48ES)” which are eight dollars for 48, I used two here to hold the puck firmly. The magnets in the puck are strong, so this provides enough adhesion to keep it in place when I pull the phone away.

Simple, cheap, and the 3M strips remove cleanly for when things change. I can now just drop my phone here as I work and it’s visible and topped off. The AirPods also work well.