True to the original spirit, each player might cause an slightly different performance to take place, as each player slightly differs from the rest of players, but the end result is similar: you hear an interpretation of 4'33".

Some players do very little overall. Some do a bit more of upfront work and then act as if the metaphorical piano lid had been closed at the beginning of the performance. Some of the upfront work briefly blocks the UI thread (not that you would notice... generally); others actively generate no sound, others do generate sound, albeit in an optimised, but equally quiet, manner.

Any fan or hard drive noises emitted by the device the player is running on, any increment in temperature, and any other external noises that might occur during a performance are all part of the performance.

Configuration

Not that a member of the audience should tell the players what to do... but there is a small configuration section that can be opened and allows to modify the length of the performance, and whether to actively emit sound.

These options are there for debugging purposes and you are encouraged to not tinker with them as they will cause a different piece to be performed, but you can tinker with them if you want to, because it's your device and you decide what to listen to.

Just make sure your device's volume is not very loud, as you might get a lot of noise blasted through the speakers otherwise!

A little bit more about the players

A classic Object Oriented Programming (OOP) hierarchy is observed as the code was written aiming to reuse as much as possible.

We start with a Base player, which is in turn extended by two categories of players: those that use an AudioContext based solution, and those who generate audio data directly and output it to an <audio> element; both categories contain multiple players, some building upon other types of players.

The UI is a web component which is instantiated per player. The UI interacts with the players by calling their play and pause methods. OOP makes this part very straightforward to implement as it defines a "shared API" for the players.

The hierarchy of players

• Base
  • No OP
  • AudioContext (and nothing else)
    • AudioContext with AudioWorklet
    • OfflineAudioContext
  • Generate WAV
    • Generate WAV with Web Worker

The players, in detail

0. Base

Keeps track of time, provides pausing, resuming and detecting end of performance functionalities.

It can also attach to an UI element, and update the element with the current playing time and status.

1. No OP

Does nothing apart from what the base player does.

2. AudioContext

An AudioContext is created, but no nodes are connected to it.

3. AudioContext with AudioWorklet

An AudioWorklet node is used to generate audio data with an instance of SilenceProcessor, which extends AudioWorkletProcessor.

4. AudioContext with OfflineAudioContext

Audio is rendered upfront using an OfflineAudioContext, then played through an AudioBufferSourceNode in the AudioContext created by the parent class.

5. Generate WAV

The audio data is generated upfront, then played through an <audio> element.

Depending on the configured length of the performance, this could considerably block the UI thread, making the website unresponsive.

6. Generate WAV (with Web Worker)

The audio data is generated in a web worker rather than in the UI thread.

This takes slightly longer to initialise, but it won't block the UI thread.

Discarded options

These are players I initially came up with but did not implement.

AudioContext with ScriptProcessorNode

My initial list of ideas included a player implemented with ScriptProcessorNode but since then a better implementation option has been standardised and shipped in browsers, so I never wrote that player. Instead, the AudioWorklet version exists.

MediaRecorder

I was not deeply familiar with the MediaRecorder API at the time I sketched out my ideas, so I wasn't super sure of how it would work in practice.

When I started implementing the various players it became evident to me that it would be a bit futile to implement a MediaRecorder based version as I don't think you can render streams offline, just as you can render audio files with OfflineAudioContext: we would need to first spend the performance time generating the stream so the recorder would record it, and only then when finished recording and the audio file was generated we would use this file with an <audio> element or another AudioContext, which would enable the user to play the generated stream. That sounded a little bit too performative, so I decided not to pursue this version.

Using other APIs or combination of APIs

I could have also built players that did things such as create oscillator nodes but also play them with a gain value of 0. This would be like having a drummer bang a drum outside an anechoic chamber, and record the performance from inside the chamber. It could be done, but I didn't, because although the end result is on the surface similar, I felt that this line of players veers off from the initial spirit as they would be actively generating sound that we then mute.

There might also be other ways that don't directly involve <audio> elements or Web Audio APIs and which I have not explored, like for example generating an image in a canvas, extracting the colour values out of the canvas element and converting those into an array of data which could be used to create an audio file or ArrayBuffer... but I felt that the point has been proved already, as this was very similar to the existing players, and it just added extra layers of complexity around them without adding a lot of interest.

There might also be other APIs that I don't know of yet or that do not exist yet that could enable new interpretations of this piece in the browser. I might revisit the project in the future.

Why build this?

One morning almost ten years ago, I was having breakfast over a reconverted sewing machine table (sans machine, that is) in the former Black Sheep Coffee shop in Charlotte Street in London—a narrow, dark ground floor unit with mismatched floor tiles—.

As usual, I was scanning Twitter to see if there was anything relevant that had happened overnight and that needed replying to (something that I did as part of my devrel job!). Then I spotted an interesting Twitter discussion about 4'33" and JavaScript that sparked the lightbulb in my brain and caused me to get my notebook out and start writing down all the ways in which the piece could be implemented in JavaScript.

I had been working a lot with audio and Web Audio stuff at that moment, so I'm not surprised in the slightest that my brain was keen on coming up with all sorts of solutions.

And then... I didn't do anything about it.

But the thought of the project would come back to me from time to time. And I still didn't do anything about it.

Until now!

In which I have built it because I am tired of thinking about building it 😆

Who?

But who prompted the idea? I hate it when I don't remember these things!

I consulted my Twitter archive to locate the thread. I more or less knew this happened in 2015 or maybe 2016... Thankfully, I didn't need to read all the tweets of that period because I still have that notebook and I found the notes!

The notes weren't dated as I just seem to have written them down so quickly, no time for formalities, but there was a date on the next page, so that helped me narrow the timeline down: no later than November 2015.

With that, I could find my side of the conversation in my archive, and who was I tweeting to, but as this is only an archive of my tweets, the full exchange is incomplete, and the tweets from the other participants, including the initial tweet that started the thread, are missing.

However this was a short interaction so I can reconstruct the context (at least from my point of view):

Darius Kazemi started the original thread, and Jenn Schiffer retweeted it. And then I read it, and enjoyed the idea, and excitedly responded with a Web Audio based version:

simplified to:
`new AudioContext();`
for performance 😜

And you know the rest!

Belated thanks to Darius and Jenn for the initial prompt 😃

These cards have barcodes or QR codes of some sort which represent your membership number. You can use your device's camera to scan the barcode and create a 'virtual' card quickly.

Please note: this is not a wallet app, and it does not deal with financial information of any kind.

You can "install" the website to your device, in which case it effectively acts like an app, or what is also known as a Progressive Web App.

Once installed, it should work offline. That is, you don't need any data service to access the cards you stored. This allows you to use the website even when there's no mobile reception and the shop doesn't offer free wifi. For example, when you reach the till of the grocery shop in the basement of the department store. Also, since store store cards does not try to find your location to recommend you "offers", it is as private and fast as it can be, and it won't make you waste your precious battery charge or time when it you're trying to scan the card and a queue of people is forming behind you...

Oh, and it is also really small (compared to a native app): the final code is less than 2 megabytes (yes, mega bytes).

Privacy

This works entirely client side. The data stays in your device. I do not know what you scan, and I do not want to know!

If you want to make a copy of your cards or want to see what is stored, inspect the website with your favourite developer tools, and look in localStorage.

I run lightweight statistics to get a sense of how many people use the website, and detect crashes or bugs. Individual visitors aren't identifiable, as I don't store any personal user data, only the fact that you visited, the type of operating system you use, etc (this is information that you provide already by browsing the internet). If your device has any sort of tracker-blocking functionality, it is likely to block my statistics code, so if the app exhibits bugs when you visit I will never know, and it's OK.

The statistics service is hosted in Europe. The data is not made available to anyone. There are no ads or any other third-party script delivered with this website.

How to install or add to your home screen

This depends on the operating system and browser you're using.

iOS

Visit the website in your iOS device, then select "Share", then scroll down to "Add to Home Screen".

Android and other non-iOS

I don't know! I don't have access to recent Android or non iOS devices. But if you figure that out, please let me know and I can update this page. Thank you 💝

How to share this app with friends and family

Send them the URL: https://storestorecards.apps.5013.es/

Help! Something does not work

My camera is broken or I don't want to / can't use it

You can enter card details by hand.

The card won't scan, I can't add it to my app!

We have tested this with UK and Spanish cards. I don't know what type of obscure configurations manifest in other countries.

You can try adding the number by hand, and playing with the different barcode types, until you get something that resembles the original card.

The scanned card looks different from the physical card

Some cards only show a subset of numbers vs what the barcode actually contains. In that case, it tends to be that the first digits are not shown in the card, but the last digits are, and tend to coincide with "your membership number".

I don't really know why—I suppose to make it easier for cashiers that have to visually verify or enter card numbers with their naked eye. If you're unsure, bring both cards with you (the 'virtual' and the physical) so you can have a backup option. Once you verify that the virtual card works, you can stop carrying the physical card with you.

Some retailers will give you a big card and "portable" cards that you can add to your keyring. I've seen they sometimes differ only by a digit, but the last x digits, which are displayed on the card, are the same. My hunch is they do it to track which format is more popular, but I haven't investigated any further.

There are no icons, logotypes or colour customisation per card

This is by design. I wanted a simple app. I don't need those.

Something else doesn't work

I only have access to not very old but not very new iOS devices. I have tested this in various computers and operating systems (macOS and Linux) and it simply worked, but who knows!

If you genuinely want to help and have an idea of what is wrong, get in touch.

If you're unhappy and want to rant, maybe just find another app or service instead—I have no time or energy to help you debug weird configurations for free. No bad blood here! You're free to use whatever you want.

Why?

Libraries, coffee shops, museums, supermarkets, grocery stores, you name it: they have membership programmes, that you prove you belong to by scanning a card when you enter (or when you leave, in the case of a store).

I was tired of carrying lots of loyalty and membership cards in my wallet, so I started using an app that allowed me to scan the cards and have a "virtual" version of the card instead.

This was convenient, but the rest was really tiresome: it wanted me to give it all sorts of permissions; it often failed to render a card that would be successfully scanned... and on top of that, it was releasing almost weekly 155+ megabyte updates for no good reason, other than "Bug fixes and feature improvements". But nothing ever seemed to change or improve!

When the app we were using "warned" my partner that it would stop working unless it were updated to the latest version, I decided that we had had enough, and I set on the task of building an alternative that would be considerate with their users, and also nimbler and much faster to use. Thus, it is my gift to both of us, and now my gift for you too! 💝🎁

As of today we both use this almost daily, often several times in a row.

It has never crashed or stalled. We find it is much faster to scan, even with some barcode scanners which are quite temperamental, thanks to our secret trick: making the barcode as big as possible (actually, this is the same trick that LiDL use to speed up checkouts!).

Technology

Since this is a website, it is built using browser technologies: HTML, CSS and JavaScript. The camera is accessed using MediaDevices.getUserMedia, as I also do in a few of my other projects already.

In addition, I am using a few libraries to plug-in gaps in implementation or just to keep my sanity:

• The Barcode Detection API, if available, or a polyfill in browsers that do not implement the API yet
• BWIP to render the barcodes
• Vite for handling the build, packaging, etc, with some plug-ins:
  • vite-plugin-pwa to help providing offlining and PWA functionalities
  • vite-plugin-favicons for handling the myriad versions of icons you have to provide in order to have ONE icon placed in the user's home screen 😂

I'm extremely grateful for those libraries!

The underneath libraries and polyfills support more barcode formats than the Barcode Detection API does, but in the interest of future compatibility I have decided to only expose the same formats as the real Web API would expose. These are listed here: supported barcode formats.

I have optimised the way the libraries are delivered to the users, by deliberately splitting code into dependency vs non dependency chunks. That way you only have to download what has changed, which is normally just my code, not libraries. This is a matter of kilobytes, so "app updates" are almost seamless: you access the app, close it and reopen it and you get a new version. This is much more considerate than native apps that will ship the whole list of dependencies over and over to users, without any respect for their time or bandwidth.

Some things I wish were possible

Although I'm very satisfied that the same code base works everywhere, regardless of the operating system or browser flavour, I find the following aspects frustrating. They put websites at a disadvantage compared to native apps:

• I can't control the brightness of the screen. This would help ensure that the barcode scans even if the user has very low brightness in their device, by temporarily increasing it (and thus, increasing the contrast between black and white in the barcodes). But there's no such API for the web. We had something like that in Firefox OS, but...
• It would also be nice to be able to control the focusing of the camera somehow. E.g. if I could for example set it to "macro" it would focus on closer distance objects, and make scanning faster. But it seems like the barcode API (and the polyfill) do a good job of detecting shapes even when slightly blurry, so not a big deal.
• I also don't seem to be able to allow camera permissions forever with iOS. It seems to ask periodically if I want to allow camera usage, even if the app is installed. I don't know if I'm missing something in the manifest. It is mildly irritating, but I can live with it.
• Finally, if you store some cards locally, while accessing the website through the browser, and then "install" the website as an app, you have to re-add the cards, as there's no way to have access to the same 'storage area'. This makes it harder to 'try before you buy', which would be a great advantage of websites vs native apps.

Is my phone screen cracked?

It is actually the screen protector that is cracked!

My phone fell off my hands the very same week that I bought it, as it was bigger than my previous phone, and I still hadn't developed the "grip" to hold it safely in place. We had just finished a run on a hot and humid July morning, I wanted to document the triumphal moment, and my sweaty hands just lost it...

It then landed, of all the places, on a particularly pointy piece of gravel, which chipped the edge of the screen protector, and created a long thin crack along the protector. The screen is fine, no pixels were harmed.

I often don't even notice the crack anymore—except now that I took these pictures!

It uses the device's camera to find the colour at the center of the image, and turns it into a music note, which is then played through the device speakers.

This is all achieved by using Web technologies such as Web Audio and the Media Streams APIs; the data processing is also done entirely locally, so no camera data is ever sent to a remote server for processing.

Inspiration

The idea to build this came after I saw there is a new documentary about Neil Harbisson coming up.

He has famously declared himself to be a cyborg, as he has a light sensor mounted on an antenna attached to his skull. The light sensor is connected to a brain implant on the back of his head, and the implant turns the colours captured by the sensor into sound.

In other words, the implant allows him to hear sound, as he has achromatopsia, a type of color blindness which means 'no colour' i.e. he sees everything in a greyscale.

The set of antenna, sensor and implant also seem to be known as an "Eyeborgs" or "Cyborg antennas". I saw mentions to other versions of this device in existence in some websites, but since I couldn't find much detail about their specifics I did not take them into account when building this experiment.

Simulation, not reproduction

Given the scarcity of technical information available, the output from this experiment is at best an approximation, but it grossly aims to produce the same sonochromatic scale that is discussed in the Wikipedia page for Sonochromatism, where the red in the colour spectrum starts at F, orange is F#, yellow is G and etc all the way to magenta which is E (for the next octave).

A small settings panel can be opened to adjust several parameters, such as the amount of transposed semitones (so the note where the scale starts can be selected), how many notes in an octave and how many micronotes to use, the gain (to alter the output volume), etc.

The generated scales are slightly different from the chromatic scales which are the type of scales we tend to think of when we talk about Western music. While "normal" music uses octaves with 12 notes (semitones) each, the Eyeborg admittedly uses 360 notes, which I interpreted as "each octave contains 36 notes", i.e. each note can be either the same value as the note in a chromatic scale, or it can be slightly off in either direction - like a third off the value of the interval between "normal" notes. I chose this option as it still was close enough to the standard note name convention and so I could display the approximate note names so that people can have an idea of which note each colour is (i.e. is it an A, or is it a C? etc)

Then, to achieve the final 360 notes they say the Eyeborg can play, the tool can detect up to 10 levels of lightness for each pure base color. This results in actually more than the stated 360 notes, as we have 36 initial values + 36 x 10 variations = 396 notes.

Potential follow up ideas

I reached a lot of dead ends and made a lot of false starts, and I also wrote a lot of exploratory code to test ideas, either because I wasn't sure they would work for what I wanted to accomplish, and at points, I wasn't even sure of what it was that I was trying to accomplish!

However, because I wanted to keep as close to the original as possible, I kept leaving many of these ideas aside, which makes the end result a touch "bare", but hopefully represents quite accurately what it is to use one of these Eyeborgs, without getting access to the original hardware or having seen it in person.

A happy outcome is that now I have a well of fresh ideas and concepts to keep digging into for further projects.

Some examples:

• it is not too noticeable because this experiment only outputs monophonic sound, but one follow up idea I had was to play more than one note at once with this type if scale and see if I encounter any interesting wave overlap effect.
• play the x common colours in an image (this is actually what I started with, when I realised that actually, the eyeborg is monophonic). This was quite spooky...
• allow to listen to colours from an image selected / uploaded by the user, or from a video. Could be useful if there is no camera in the device or if permission can't be granted!
• use different types of oscillator waves
• use colour properties to alter parameters in the output such as filter cut off frequencies (I am not using a filter yet but it could be something to experiment with)

I also found a lot of things to write about including, but not limited, small bugs and Web Audio rendering differences between engines, processing colours, and much more, which I will be writing about in my blog and linking to from here.

In the meantime, you can look at the other references below or any of my other experiments!

References

• Neil Harbisson in Wikipedia.
• the Cyborg Foundation dedicated to promoting cyborg rights.

More info.

A base image is corrupted randomly at regular intervals and the results displayed on screen.

Simple demo of WebRTC + getUserMedia + Animated_GIF.js.

---

Here's the source code.

Of course it didn't take long to convert Heather's avatar into an animated thingie...

Here's the source code, just in case you want to check it out. It's basically Canvas + Tween.js.

Made for DemoJS 2013 / Paris 28-30 June.

~~~

I wanted to watch it again but the binary version was for Mac G4's. I didn't have a Rosetta environment and I didn't want to set up a compiler or any of that either, so I decided to liberally port this to JavaScript, using the work-in-progress Sorollet.js port that I had already started.

The source code is on github.

It also uses Three.js and Tween.js

HINT: Try pressing 'd' during playback for getting access to the debug panel and the synthetiser configuration dashboard. Alternatively, view the source!

More info.

GO ON, try it!

More info.