More detailed, authoritative, and up-to-date information can be found in the Introduction to WebKit.
This is a follow up to my post on Building Webkit. Here, I’ll cover how to fix and test web-facing features in WebKit.
Naturally, first find the file that relates to the feature you are trying to fix (e.g., Navigator stuff would be in Navigator.cpp, etc.).
A great search tool to help you find where things are, and to follow code symbols is SourceGraph.
Tests
Like with any software, once you’ve prepared a patch, and you’ve built, you will need to test it to make sure it works and you haven’t regressed any existing code.
In WebKit, tests are primarily in the LayoutTests/ folder. Web platform tests can be found in:
LayoutTests/imported/w3c/web-platform-tests/ – as the name suggests, imported from Web Platform Tests
LayoutTests/http/tests/ – these are modified tests to run on WebKit’s testing infrastructure over HTTP(S)
LayoutTests/fast/ – more tests!
So you need to make sure you check in multiple locations.
To run the tests, you use /Tools/Scripts/run-webkit-tests script. There are a few flags to be aware of:
You need to specify the “target” for the test runner, which, if you followed my last blog post is --debug.
The path can’t include “LayoutTests/” (see below)
Important! If you test changes the result or adds new things, you need to add --reset-results. This will reset the results with whatever your run of the tests produces/changes.
At the time of writing, http(s) tests are restricted 127.0.0.1:8443 and localhost:8443, so a lot of WTP tests won’t work.
The test development process is fairly straight forward. Poke around to see what libraries are available.
Submitting your patch
Once your tests are passing, you are ready to send up a patch to WebKit!
Firstly, make sure you have filed a bug for the thing you are adding or fixing!
Then, to send your patch up to WebKit:
./Tools/Scripts/webkit-patch upload
It will ask you to provide some relevant information and generate various change logs for you. Check that they look ok.
ProTip: I don’t recommend committing these changelogs to your working branch. If you rebase with the main branch, it makes a bit of a mess. They are auto-generated anyway, so just stash them or “git checkout path/to/changelog” them.
That will start the start the tests for various platforms (e.g., iOS, MacOS, etc.). If you are lucky, all tests will pass. If any fail, you will get an email with links to logs where you can find out what failed.
ProTip: make sure your errors spit out detailed messages when they fail. Otherwise, it will be difficult to know/where things have gone wrong (you often won’t get a stack trace, specially form JS).
If any of the platforms fail (specially legacy ones, like Windows), you might need provide new text expectations. These are found in:
Getting WebKit (the web engine the powers Safari) up and running is super easy, but some aspects are not super well documented. But once you get all the bits in place, getting a custom build of WebKit up and running on MacOS is a breeze.
Note: webkit.org provides a detailed getting started guide. It provides a bunch of additional/useful details, so please go there for authoritative information.
Getting the code
Getting the code for WebKit is easy (but please see WebKit’s instructions in case this blog post is out of date).
Note: WebKit is quite a few gigabytes… don’t expect a quick download!
git clone git://git.webkit.org/WebKit-https.git webkit
cd webkit
Building
Thanks to the provided scripts, building is also straight forward:
./Tools/Scripts/build-webkit --debug
On my 16″ MacBook Pro, this takes about 20 minutes. Note, if you have ccache installed, you can add the --use-ccache. That may improve re-compile times.
Tip: if you are doing a fair bit of hacking on WebKit, it’s useful to add /Path/To/WebKit/Tools/Scripts/ to your PATH.
Running WebKit in Safari
Running a freshly compiled WebKit in Safari (i.e., it uses your locally built WebKit as the rendering/JS engine for the system’s Safari installation) can sometimes fail when WebKit’s version gets out of sync with the installed version of Safari.
This happened to me, when I updated to Safari 14. Regardless, it’s worth a shot because it’s nice to see WebKit running in Safari proper:
./Tools/Scripts/run-safari --debug
As expected, the --debug flag compiles a debuggable (unoptimized) version for WebKit.
If you get a bunch of permission errors creating directories, you might need to “sudo” – but obviously, don’t then go to random websites. Alternatively, you can also try adding a --no-saved-state flag, which won’t write to so many directories.
Running “Mini browser”
What seems to be the preferred way to run WebKit by the WebKit community is to use what is called the “minibrowser”. As the name implies, the minibrowser is just a little browser chrome shell with stripped down functionality:
./Tools/Scripts/run-minibrowser --debug
As expected, the --debug flag tells the running which build to use. Don’t forget to add the --debug that, or it won’t work.
The minibrowser is good enough that you can run your WebKit build and do whatever testing or debugging you need to do.
Over the last month I dove into Apple’s Application Development Platform, which provides a unified set of APIs for developing software for MacOS, iOS, WatchOS, tvOS, etc.
As a Web Standards Engineer who spends most of his time thinking about and working on the Web Platform, it is rare that I would get such an opportunity to see how things work elsewhere… and let me tell you, it’s been quite a treat and really eye opening.
Note:in this post, I’m just glossing over what I learnt over the last month for my own benefit and to gather my thoughts. I intend to do a followup in which I’ll pivot back to how all this relates back to the Web Platform.
iOS Development bootcamp
My learning took place via Udemy’s “iOS 13 & Swift 5 – The Complete iOS App Development Bootcamp“, which is taught by a phenomenal application developer called Dr. Angela Yu. If you are interested in taking a similar journey, I highly recommend her course. But note that it takes about 3-4 weeks full time to complete, and only scratches the surface of what’s possible on Apple’s platform.
As the name suggests, this bootcamp was iOS centric. However, I got to build around iOS 20+ different iOS/MacOS applications (see the full PDF of course syllabus). As of MacOS 10.15, it’s possible to compile and design iOS apps a MacOS apps and they just work out of the box. So by just checking a checkbox, it was possible to run all the apps directly on my Mac.
Nevertheless, it was also cool to just hook up my iPad and iPhone and with one click have my apps running on the phone. And to be honest, it was often easier than getting things set up to run in a web browser.
Xcode
It’s a beast of an IDE.
The course covered most of the basics and getting around. Xcode has GitHub integration, as well as standard git support built-into its UI. However, it’s nowhere near as good a code editor as VSCode so I often found myself having to jump into VSCode to do things.
Working with Xcode’s WYSIWYG application layout tool was an interesting development approach. From my web experience, I thought it would have sucked because it reminded me of something old school like DreamWeaver. But it was actually quite ok.
The backing representation of xCode’s “storyboards” are a XMLs document… which is some twisted way, it’s like HTML although you never really interact with this document directly. These XML “storyboard” documents then link to code, via “IB” (interface builder) outlets and actions. These are similar to event handlers or just using global variables on a web page. For example, “@IBAction func addButtonPressed(){}” would handle a button press.
Example of Xcode visual layout together with Code.
Swift
People who known me know that I’m a huge Swift fan. It’s without doubt my second favorite language after JavaScript – and gained even more respect for it throughout this course. Although the course didn’t cover every aspect of Swift (e.g., it didn’t go into the super powerful enums of Swift), it covered enough to be able to use all the APIs that were covered in the course.
Things I really like about Swift:
protocols and extensions in general
Data types
Easily insert variables “like by \(doing) this”… (yeah, I’m looking at you, Rust).
explicit argument names and aliased arg names. This makes for super nice, legible code.
immutable structs + passed by value
“let” and “var” only
enums and their shorthands.
switch/case behavior
Optional clousure declaration on final arguments
The do {} catch blocks, with explicit `try` declarations on functions calls. That way, one can explicitly see when a function will throw in code.
Automatic variable name generation for things, like “error” inside a catch block.
Things that Swift desperately needs or don’t particularly enjoy:
lack of async/await. The reliance on callbacks is very JavaScript around 7 years ago.
Building `if let x` pyramids of optional types.
It’s an amazing language that feels like scripting language and makes total sense to anyone who knows JS or TypeScript. But it’s a full fledged industrial strength no-messing-around proper programming language.
UI Library
The UI component library is rich and extensibly focused on application creation. Although there are similarities to what one finds in HTML with regards to common accessible form controls (buttons, text input fields, etc.), the platform provides a significantly richer set UI elements. It also provides means of extending all the provided components. This is similar to “Web Components” in a sense.
Xcode allows you to drag and drop UI components. You just search for what you need and drop it in place.
UIKit
Extremely good for creating application UIs, but the layout constraints system can be quite frustrating and I imagine takes a lot of practice to get used to. I guess it must feel the same using CSS Grid or FlexBox until you “get it”. However, I did appreciate the simplicity of just having to think about everything in terms of horizontal, vertical, and “Z” stacks when designing layouts and filling in space. That pretty much the same a CSS grid and FlexBox.
Example calculator showing horizontal and vertical stacks to do layout.
Documentation
Apple’s documentation is, as expected, fairly good. However, I think unlike a lot of documentation around the Web Platform, it does assume a lot of prior knowledge and can be at times challenging to decipher. At minimum, you are assured that there is some documentation for everything, even if at times it seems that it was automatically generated.
And let me tell you… having some documentation has been nice, specially coming from other open source projects where documentation is often non-existent.
MVC and Delegate model
The “Model View Controller” (MVC) design and the “delegate” pattern feature heavily throughout Apple’s platform. A delegate is essentially an object that can handle actions generated by other object – so you “delegate” actions, like, say, providing data to a data table, to another object using a standard “protocol”.
Although I personally find the delegate model somewhat fragile (as opposed to a subscriber model), it came to appreciate its consistency throughout Apple’s platform.
Networking and JSON processing
Personally, I think if a platform makes it difficult to fetch and process JSON over HTTP, it’s dead in the water. Thankfully, Apple makes it easy to spin up a “URLSession” to fetch data. I also really like that you can’t fetch insecure HTTP resources at all: you have to use HTTPS. Processing JSON is also fairly straight forward, but Encoding/Decoding (or making structures that represent JSON “codable”) can be quite tedious.
Thankfully, they’re are libraries like SwiftyJSON that make encoding/decoding arbitrary JSON data quite easy… though obviously not as nice as working with straight up JSON in JS in a browser or node.
SwiftUI
Swift UI was quite a treat. It again mixes layout and functionality, which can become a bit of a mess, but because of the simplicity of its layout model (HStack, VStack, ZStack), it makes it quite easy to lay out applications – and to control how items in each stack are distributed. Unlike with UI Kit, it was also nice to have direct control over everything without relying on Xcode’s WYSIWYG… but at the same time, you have to dig around to figure out where certain properties are hidden within various object structures.
Example application that loads up hacker news items and displays them in a UIWebView.
The @state and @published property modifiers where super nice, and something missing from the Web. It’s nice to just be able to set a custom variable/property and be assured that the UI will be updated automatically at the right time.
(Google’s) Firebase
Although not an Apple technology, it was interesting to play around with a third-party storage and communication service for building applications. In the course, we used Firebase to make a realtime chat application.
Storage
For storage, it was interesting to see the variety of different storage options available on the platform. These ranged from using plists, saving to files, using Core Data, using Realm, which is a third party database owned by MongoDB.
Core Data felt very ol’ school, while Realm was quite nice to use. I also found the defaulting to synchronous read/write operations with the Apple solutions somewhat concerning. Real seem to handle async operation much better.
Swift package manger
Unlike NPM, I kinda liked the Swift package manager’s direct integration into Xcode. In Xcode, you can simply point to a GitHub repository, and Xcode then figures that it’s a library extension.
Unlike using Cocoapods, SPM then sets everything up for you in Xcode without needing a separate project.
In-App purchases
It nice to see how in-app purchases are set up both within an application and also via Apple’s developer portal. It was also interesting to see how in-app purchases are classified:
Consumable – buy it, use it once (e.g., some kind of power-up).
Non-consumable: buy, and keep it (e.g., a sword)
Auto-renewable subscription, like a magazine subscription.
Non-renewable subscription: like access to a back catalogue for a year.
CoreML and CreateML
This was my first exposure to machine learning, so I found it quite fascinating and I was pleasantly surprised at both how easy the API was to use, and how easy it was to train a model. For images, you literally just create folders, drop in some files, and click a button in the CreateML application:
Data used to train how to recognize different animals: dog, cat, rabbit – along with data used to verify if the training worked.
At the same time, it was quite laughable how limited this technology is if not trained correctly (or without really understanding what it’s actually doing under the hood). And I have a fear that badly trained models will lead to really bad things happening in the world. Admittedly, I had a great time running around the house with my iPad (wrongly) identifying different plants… or identifying my cat as a “cat”.
Concluding thoughts
To say that Apple’s platform is anything but super impressive would be an understatement. Every API is filled with potential to unleash a mass of creativity, and Swift is just the right language through which to express that creativity.
Having said that, buying into Apple’s platform comes at a cost (a quite literal one!). However, I’d like to cover the tradeoffs, and how all this relates back to the web, in a followup post.
For now, it was nice to be a blank slate and just have the opportunity to explore a new terrain.
