I use macOS a lot for various tasks. I have tried doing everything on Linux, but it always becomes an issue, especially with Linux on the desktop and ecosystem as a whole. But all of that is for another post.
I run a batch job that uses Selenium to scrape a report out of a system I use at work. Why do I not just use an API to get the data, you ask? I wish I could, but as a company grows and gets a larger market share, one of the first things to go seems to be an API. I suspect this is to ensure you are stuck in their walled garden.
Fine, I can work around it by scraping the data I need from their system.
I use Homebrew on macOS as my package manager, and in order to use Selenium, it is necessary to use a webdriver. I used to use Mozilla GeckoDriver but had some issues. It was a long time ago, and I don’t recall all of the issues anymore, but I switched to ChromeDriver. No problem, really. I just do
brew install --cask ChromeDriver
and it is installed. Straightforward. If only it was so easy.
First problem…
macOS ships with Gatekeeper. Gatekeeper is a security feature to prevent the installation and execution of malware. That is a good thing. But it enforces code signing. This means when you run an app for the first time, the system verifies the application has been signed and is safe to run. If not, it prompts you to delete it. Again, this is a good thing, but I run the job that utilizes the ChromeDriver in a batch job. This means that every time it is upgraded, it has to be reauthorized on the system. The usual way to do this is to go to System Settings > Privacy & Security and select Allow Anyway.
This is problematic when I run the job from cron as a batch. If it was infrequent, no problem. But the ChromeDriver must be updated every time the Chrome Browser is upgraded.
I needed a better way to do this so it happens automatically.
No problem, the system ships a utility called xattr. This utility allows me to change the extended attributes on the executable that Gatekeeper uses to determine if an application is quarantined or not.
Second problem…
That seemed like the perfect solution. I would use xattr to clear the extended attribute and Gatekeeper would be happy and my batch job would run. The problem is that if you check where the application chromedriver lives with the which command it returns /opt/homebrew/bin/chromedriver. Unfortunately, that is a symbolic link to the actual application so clearing the extended attribute on it would not work. I need to clear the extended attribute on the actual application.
Third problem…
Simple solution, let me just check where the symbolic link points to and change that file.
ls -l /opt/homebrew/bin/chromedriver -> /opt/homebrew/Caskroom/chromedriver/132.0.6834.159/chromedriver-mac-arm64/chromedriver
Turns out that won’t work because every time chromedriver is updated the path changes. That is a problem.
Simple solution, I will just use the readlink utility to grab the actual path from the symbolic link for my script to clear the extended attribute.
Fourth problem…
The final issue I had was I needed to know if chromedriver has been updated or not. I am sure that a large array of solutions to this exist. Mine was to simply check the current date of the file and see if it has run since the last time I checked. The ls command on Linux has a –time-style option that will output the file date in a format that is consistent and easy to compare. One issue though – Linux uses the GNU ls command. MacOS doesn’t. So no –time-style option.
To fix this, and many other issues, it is best to install the Homebrew Coreutils package to get the GNU utilities that are the same as those on Linux.
brew install coreutils
Final Solution…
I put all of this information together and drop it into a zsh script (zsh is the MacOS default and I try to use stock everything when possible). Homebrew does not link the Coreutils utilities in order not to interfere with the built in OS utilities. When you want to use one of the GNU tools installed with Coreutils just prepend a g to the command.
#!/usr/bin/env zsh
target_date=$(gls -l --time-style="+%Y-%m-%d" /opt/homebrew/bin/chromedriver | awk '{print $6}')
yesterday=$(date -v -1d +%Y-%m-%d)
if [[ "$target_date" > "$yesterday" ]]; then
xattr -d com.apple.quarantine $(readlink -f /opt/homebrew/bin/chromedriver)
fi
I get the date of the application, check yesterday’s date (I will run this daily before my batch runs). If the application has been updated since yesterday, I then remove the extended attribute. Gatekeeper is now happy, and my batch will run.