p14nd4's blog is the best blog ever.

I’ve recently been using self-hosted Gitlab at work, particularly for its CI/CD features. We have a few projects using it already, and I’m trying to expand that utilization. It all went pretty well until I tried building with a new Docker image, and got an error:

ERROR: Job failed: failed to pull image "[..]:latest" with specified policies [always]: Error response from daemon: Head "https://[..]:5050/v2/[..]/manifests/latest": denied: access forbidden (manager.go:237:0s)

The project I most recently set my sights on requires some external tools/resources for the build. While they’re publicly available, the prospect of adding a build script step to download and extract them was unpalatable (waste of bandwidth and time), and while I toyed with the idea of trying to hack them into build cache restoration, I finally relented and just built a new Dockerfile / docker image, and pushed the image to the project’s container repository. (Incidentally, I would’ve loved if .gitlab-ci.yml understood Docker Compose, and I could just mount a volume containing these resources on top of an upstream-maintained/updated Docker image, so please let me know if I missed that or it gets added.)

The Problem

All of that got me to the point of trying to run the build for the first time, which failed quickly with a log like this:

Running with gitlab-runner 16.2.0 (782e15da)
  on gitlab-runner02 Ua6jigiQ, system ID: r_tRd8aLibbXym
Preparing the "docker" executor 00:03
Using Docker executor with image [gitlabHostName]:5050/[project/image path]:latest ...
Pulling docker image [gitlabHostName]:5050/[project/image path]:latest ...
WARNING: Failed to pull image with policy "always": Error response from daemon: Head "https://[gitlabHostName]:5050/v2/[project/image path]/manifests/latest": denied: access forbidden (manager.go:237:0s)
ERROR: Job failed: failed to pull image "[gitlabHostName]:5050/[project/image path]:latest" with specified policies [always]: Error response from daemon: Head "https://[gitlabHostName]:5050/v2/[project/image path]/manifests/latest": denied: access forbidden (manager.go:237:0s)

After trying several things and banging my head on the wall a bit, I eventually spotted a difference between this and a working build: the presence of this line before ‘pulling docker image’:

Authenticating with credentials from job payload (GitLab Registry)

The Solution

Additional searching and scrutinizing the config revealed that in a working build, the image repository path hostname was lowercase, and in the new/broken build, it had capitalization. Sure enough, when I replaced the “gitlabHostName” with “gitlabhostname” (equivalent), the new build started “Authenticating with credentials from job payload (GitLab Registry)” as well, and was able to complete the build.

So, if you’re butting your head against Failed to pull image … denied: access forbidden errors in Gitlab while trying to pull an image from your container registry, check to make sure the hostname specified in the .gitlab-ci.yml image path matches your actual Gitlab container repository and is lower-case. (After seeing this only work with the lower-cased hostname, I was surprised to find that the value in my gitlab.rb registry_external_url does have the capitalization found in the broken build, so it would seem that it’s not even a matter of matching that value, but necessarily being lower-case.)

I’ve recently been debugging an old project I inherited that features a [mostly] Java server and C++ client. I think it was mostly written 10–15 years ago (Java 6 at the latest), and probably not touched a whole lot since then, so while it’s all too easy to complain about any code, let alone old code, and code written by anyone else (and I will), I do also have respect for the fact that it’s moderately complex and has generally worked for a long time.

Although there’s plenty of low-hanging fruit for improvement in the project, the killer has been an issue seen when the system isn’t started up or shut down in its preferred sequence and speed. I never identified a perfect recipe for reproducing the issue, but basically: sometimes the system would get into a state where some clients would connect, but then the server seemed to essentially stop letting anyone new in. The workaround was to stop all clients (luckily the clients in this case were actually other servers within our [customers’] control), stop the server, start the server, and then start up the clients one by one. This is annoying with four servers; it’s a major pain with 40. (For better or worse, nobody currently has more than that.)
Read more »

I just spent a while fighting an exception that was being thrown from within a .NET runtime-generated assembly—specifically from Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationWriter{MyClassName}.Write7_{MyClassName}. There were plenty of search results that didn’t apply to my problem, but I didn’t find any that directly addressed the issue I was encountering, so I wanted to write up a quick post to possibly save someone some time in the future.

Read more »

After pfSense upgraded to a newer FreeBSD base (perhaps a year ago), my Thinkpad T41 running pfSense was failing to boot; I believe the errors were something about failing to initialize the device at ata1 (the optical drive); perhaps something about failing to respond to a command. After lots of searching (and poor documentation), I resolved the issue by setting this boot parameter:

hint.ata.1.disabled=1

Once the system booted, I believe I added this to /boot/loader.conf.local to make the fix [slightly more] permanent. (I believe this file still gets wiped out during system updates, so it’s not a perfect solution.)

I’ll update this post with the exact error message if I encounter it again. (I’m finally getting around to writing this post since I’m about to wipe pfSense and try OPNsense, and I don’t want to have to re-discover this solution.)

Edit: As I suspected, OPNsense has the same problem:

(ataprobe0:ata1:0:1:0): ATAPI_IDENTIFY. ACB: a1 00 00 00 00 40 00 00 00 00 00 00
(ataprobe0:ata1:0:1:0): CAM status: Command timeout
(ataprobe0:ata1:0:1:0): Retrying command
(ataprobe0:ata1:0:1:0): ATAPI_IDENTIFY. ACB: a1 00 00 00 00 40 00 00 00 00 00 00
(ataprobe0:ata1:0:1:0): CAM status: Command timeout
(ataprobe0:ata1:0:1:0): Error 5, Retries exhausted

For a demo project I had at work recently, I wanted to incorporate some imagery from the client’s web site. I discovered their web server was configured to generate an index page for the directory containing their images, but I didn’t want to click through each one individually to figure out if it was what I wanted. (An auto-generated index page is something like this, but imagine it having hundreds of entries.)

I was pretty sure that I’d encountered some javascript many years ago that would render a thumbnail gallery for just such an occasion, but I was unable to locate it. Thankfully, I’m a software engineer, and I’m easily distracted by tangential pursuits, so I just wrote my own! If you’re not a nerd, but want to see what it is, copy the text below, visit the index page mentioned above; in your browser address bar, delete the address that’s currently there, manually type javascript: (make sure to get the colon), and then paste the text from below. (Browsers strip the javascript: from the text if you paste it.)

If it works (tested in Chromium 47 and Firefox 43), you should be presented with a little gallery of images! You can also search the web for index of jpg or something similar and test it out there, but you can stumble across some…interesting…things that way. You’ve been warned.

Also, it’s got the potential to crash your browser (or at least really slow things down) if you run it in a really large index. For better or for worse, I don’t do any intelligent loading/unloading of images, so it just ends up rendering a page with ALL OF THE IMAGES from the index. I have no doubt that this can be dramatically improved in a variety of ways; I very infrequently touch JavaScript, so this was a bit of a refresher exercise for me. Pull requests welcome!

One of the sites I run, Techware Labs, uses WordPress for its CMS. (As an aside, we previously used our own home-grown CMS, which had a variety of benefits, but didn’t provide quite as much flexibility as we wanted. Rather than putting significant effort into rewriting or massively upgrading it, we decided to leave that heavy lifting to the pros.) We began using WP around May, 2009 (v2.7.1), since which point 17 major releases have occurred (and significantly more minor releases). We selected the Multi-Page Toolkit plugin to paginate articles (since WordPress just sticks the entire post on a single page by default), which seemed reasonable at the time—it was relatively popular, and had been updated a few months prior. Unfortunately, that update a few months prior [to early 2009] is the last update the plugin has received at the time of this writing.

A lot can happen in seven-plus years, especially in a popular software project like WordPress, but miraculously this plugin mostly kept working through it all. Mostly, however, isn’t quite the same as entirely, and some people get cranky when one thing goes wrong. (Take the Hindenburg, for example; why doesn’t anyone talk about its 62 successful flights?)

Read more »

If you ready my previous post, you already know that I’ve had frustrating issues with lock-ups in Ubuntu 12.04 on my new Dell OptiPlex 9010, and perhaps related oddities with errors reported in memtest86+. I opened a support ticket with Dell, and they shipped me a replacement system, which arrived today. The spoiler: it didn’t fix my problems. However, in the course of determining that fact, I may have further isolated the issue.

Read more »

The History

I’ve been waiting for a while to get a new desktop computer at home. Back in 2006, I was biding my time to replace my dead desktop until the Windows Vista buy your computer now and we’ll give you Vista whenever it comes out program to be officially announced by Dell, and finally bought my OptiPlex 745 Small Form Factor PC (with a nearly top-of-the-line Conroe Core 2 Duo E6600 CPU) in October. Since that date, I’ve been running said computer nearly 24 hours a day, seven days a week, and it’s held up remarkably well.

While I ran Windows XP full time when I got it, I switched to Ubuntu full time (aside from playing through Portal when it came out) whenever I got my hands on Vista. That’s not to come as a condemnation of Vista, as everyone is so quick to do, but it just worked out that way: since I was going to do a fresh install on a fresh hard drive for Vista, it was a perfect opportunity to install Ubuntu alongside. And once Ubuntu was installed, I wanted to keep using it (as I had been full time in the months between my old desktop dying and purchasing that new computer). I preferred the desktop environment, window management, and most of the apps were either the exact same ones I used already on Windows (e.g. Opera, Firefox, Pidgin), or had suitable (if not preferable) replacements, with the exception of Foobar2000 and Media Player Classic (MPC). Things worked well, too!

Read more »

I was very disappointed to see today that Motorola announced it’s effectively decided to abandon the Admiral and its user-base less than seven months after releasing the product (though, I’d imagine the decision was actually made long ago). I hope that by not not even providing an upgrade to the version of Android (4.0, Ice Cream Sandwich) that was released[1] prior to the Admiral’s retail availability[2], Motorola’s message to current and potential customers is heard loud and clear: the relationship ends once your credit card clears from buying the phone.

While the Admiral had the potential to be a great phone, this should pretty much condemn it to failure, since anyone who knows anything will know better than to buy it now. And I, at least, now know not to ever buy another Motorola phone and expect anything more than whatever software it ships with.

(Mostly a copy of my post on Motorola’s forums, but I’m not sure if they’ll leave the comment up or delete it.)

Citations

• [1] Android 4.0 released October 19, 2011
• [2] Motorola Admiral released October 23, 2011

I wrote my initial review of the Motorola Admiral (my new phone) about three and a half days after getting my hands on it. I’ve been using the phone full-time for almost two weeks now, so wanted to follow up with a few other thoughts. You should definitely read the initial review above to put these additional notes into context.

Read more »