Elroyjetson

Home > Notes

Automatically Download New YouTube Videos

Posted on: by

I was on Twitter yesterday and ran across a couple of posts by Federico Viticci where he describes a workflow he has created that allows him to automatically download YouTube videos to a server he owns and then uses Zapier to notify him when new videos are downloaded via a Pushcut notification on his iOS device.

Apparently a lot of people have been asking him to write up an article on how he does all of this. Federico is busy with Macstories and keeps pushing it off so I thought I would write up how to do the server side of the download. This is going to use youtube-dl, bash, and launchd. You can switch out launchd with cron if you are not on a macOS system but I was recently forced to convert my home Linux server to a gaming machine for my son and now my home server is on a Mac mini that it replaced.

A caveat if you are on Linux, download youtube-dl directly instead of using your distro package. It updates often and it can become unusable if you can’t keep up.

On macOS, I install both Bash and youtube-dl using Homebrew.

A caveat if you are on macOS Catalina, Apple has switched from Bash to Zsh as the default shell. You can change this back to Bash fairly easily just make sure you are pointing to the Bash version installed by Homebrew which is usually at /usr/local/bin/bash.

While macOS doesn’t utilize the XDG Base Directory Specification, I am going to use it simply because it keeps my home directory neat and tidy and it makes it more compatible with running this on a Linux server. Basically it means that I will be using a directory called .cache to store some information that the script will be needing.

The basics of what we are going to do is take a list of YouTube channels and feed them to youtube-dl which will check if any videos exist and if so will dump them to a directory we specify (in my case to my Plex server), along with the video description, caches the video it downloaded so it doesn’t try to download it again, and then writes a log of what was down to a file for debugging purposes.

Federico uses Dropbox which he then watches with Zapier to kick off the notification portion of the workflow. While I will not cover the notification portion of what he does, you could probably simply use Gmail along with Zapier and edit the bash script so that the last thing it does is send the email.

Downloading videos will take up a good deal of disk space really quickly so you probably want to limit what you download to only the channels that you watch and don’t want to miss. The first thing you need is to get a list of the channel URL’s. For this example I am going to use Tech Craft (https://www.youtube.com/channel/UCT-GpMtIFhX9EMA0Eauevhw) and 512 Pixels (https://www.youtube.com/channel/UCZzXBTOSdtmOz9_VMYffr4g). You can have as many in your list as you want.

First thing I want to do is build out the youtube-dl command.

1: youtube-dl "$i" \
2: --dateafter now-1day \
3: -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' \
4: --download-archive ~/.cache/youtube-dl/youtube-dl-seen.conf \
5: --write-description \
6: --output "/Volumes/Ext1/Videos/YouTube/%(uploader)s/%(upload_date)s - %(title)s.%(ext)s"

Note that the #: is not part of the command but just makes it easy for me to reference it here.

Line 1: We will replace the $i with the YouTube channel we want to get videos from.
Line 2: This line tells youtube-dl to only get new videos in the last 24 hours (there might be a better way to do this)
Line 3: Since my plex server runs on macOS I want to get the best quality video and audio that is in an mp4 format.
Line 4: I want to keep track of the videos that I have downloaded so I don’t download them again. This says to cache the videos in the specified directory.
Line 5: This line tells youtube-dl to also download the video description and save it in a text file along with the video.
Line 6: Download the video to the specified directory (in my case it is my Plex server directory) and in a directory for the channel, name it by the upload date and video title.

With that complete we simply create a bash script that will loop over all of the YouTube channels and then utilize the command we just constructed to download the videos.

#!/usr/bin/env bash

array = (
    "https://www.youtube.com/channel/UCT-GpMtIFhX9EMA0Eauevhw"
    "https://www.youtube.com/channel/UCZzXBTOSdtmOz9_VMYffr4g"
)

for i in "$[array[@]}"
do
    /usr/local/bin/youtube-dl "$i" \
    --dateafter now-1day \
    -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' \
    --download-archive ~/.cache/youtube-dl/youtube-dl-seen.conf \
    --write-description \
    --output "/Volumes/Ext1/Videos/YouTube/%(uploader)s/%(upload_date)s - %(title)s.%(ext)s"
done

Finally, setup a launchd agent to run the script once per day. To do this we will create a plist file and save it into ~/Library/LaunchAgents. This plist file will tell launchd when to run the script (11pm daily), where the script is to run, and where to log any output.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>local.DownloadYouTube</string>
  <key>Program</key>
  <string>/Users/jking/bin/yt.sh</string>
  <key>StandardOutPath</key>
  <string>/Users/jking/.cache/yt/yt.log</string>
  <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>23</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
</dict>
</plist>

Finally, load the Launch Agent to start it running.

launchctl load ~/Library/LaunchAgents/org.elroyjetson.yt.plist

To download any of the code go to the project on Github.

Updated: