jaseg/blog
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

deploy.py auto-commit

Browse source
This commit is contained in:
jaseg 2021-08-16 14:26:57 +02:00
commit 28596aeee9
162 changed files with 3177 additions and 3267 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -1 +0,0 @@
public

0
.gitmodules vendored
View file

77
404.html Normal file
View file

@ -0,0 +1,77 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>404 Page not found | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
404 NOT FOUND
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

126
about/index.html Normal file
View file

@ -0,0 +1,126 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>About jaseg | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<div class="article-meta">
<h1><span class="title">About jaseg</span></h1>
<p class="terms">
</p>
</div>
<main>
<div class="document" id="about">
<h1 class="title">About</h1>
<p>Hej, I'm Jan, or jaseg. At the moment I'm doing a PhD (Dr.-Ing.) at TU Darmstadt in Computer Science, specializing on
Hardware Security. This is my personal website where I publish things that I find interesting.</p>
<p>I self-host my code at <a class="reference external" href="https://git.jaseg.de/">git.jaseg.de</a>, but I am also on <a class="reference external" href="https://github.com/jaseg">github</a>
and on <a class="reference external" href="https://gitlab.com/neinseg">gitlab</a>. I use github for issue tracking for some of my projects such as
<a class="reference external" href="https://github.com/jaseg/gerbolyze">gerbolyze</a> and <a class="reference external" href="https://github.com/jaseg/python-mpv">python-mpv</a>. I maintain
the <a class="reference external" href="https://pypi.org/project/python-mpv/">python-mpv</a> and <a class="reference external" href="https://pypi.org/project/gerbolyze/">gerbolyze</a> python
packages on PyPI. Release tags on these two repositories are signed with the release signing key found <a class="reference external" href="https://github.com/jaseg.gpg">on github</a> and below.</p>
<p>I am not on any social network, but feel free to write me an email at <a class="reference external" href="mailto:hello&#64;jaseg.de?subject=About page on blog.jaseg.de">hello&#64;jaseg.de</a>.</p>
<p>I do not use application-level email encryption such as S/MIME or PGP. If you need a higher level of secrecy than
regular old email provides, please ask around for my signal contact or email me a file encrypted using <a class="reference external" href="https://github.com/FiloSottile/age">age</a> with one of the SSH keys listed <a class="reference external" href="https://github.com/jaseg.keys">on my github</a>. You can find both PGP and other SSH keys that I have used in the past on the
internet, but please consider these keys revoked, and do not use them to encrypt anything you send me.</p>
<div class="section" id="python-package-release-signing-key">
<h2>Python package release signing key</h2>
<p>I use this GPG key (key ID <tt class="docutils literal">ED7A208EEEC76F2D</tt>) to sign git release tags of both <a class="reference external" href="https://github.com/jaseg/gerbolyze">gerbolyze</a> and <a class="reference external" href="https://github.com/jaseg/python-mpv">python-mpv</a>:</p>
<pre class="literal-block">
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEXom49xYJKwYBBAHaRw8BAQdA/KrWMt2MKGIZUvlQZnWjNd6i8/ZYjRsBQqEf
PJ8pJ+20NHB5dGhvbi1tcHYgUmVsZWFzZSBTaWduaW5nIEtleSA8cHl0aG9uLW1w
dkBqYXNlZy5kZT6IlgQTFggAPhYhBONvdTB/Cg7C0UX/XO16II7ux28tBQJeibj3
AhsDBQkSzAMABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEO16II7ux28thRYA
/3Yl1RdeUGor6K0RTxce9TIBB+DpLNupJgB9f6onuocpAQC614zQ/RQ6rkGTHCwA
ElFClWRQ5eppj0jpAuH15udqAbg4BF6JuPcSCisGAQQBl1UBBQEBB0A0mrXSv6rj
ajCmZR4H4OtowAx477YS+yWARqo1NtdgJwMBCAeIfgQYFggAJhYhBONvdTB/Cg7C
0UX/XO16II7ux28tBQJeibj3AhsMBQkSzAMAAAoJEO16II7ux28tMZwBAIUpHHvP
gRW2jQuzdw1r06kItfFk/0t+mgNUQ2+vtbhzAP98BoWx7lv+bvlIbBaVgLldusj0
pHnZI/0y3ksMBkdbBw==
=Mr6G
-----END PGP PUBLIC KEY BLOCK-----
</pre>
</div>
</div>
</main>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

6
archetypes/default.md
View file

@ -1,6 +0,0 @@
---
title: "{{ replace .Name "-" " " | title }}"
date: {{ .Date }}
draft: true
---

81
categories/index.html Normal file
View file

@ -0,0 +1,81 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Categories | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<h1>Categories</h1>
<ul class="terms">
</ul>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

10
categories/index.xml Normal file
View file

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Categories on jaseg.de</title>
<link>https://blog.jaseg.de/categories/</link>
<description>Recent content in Categories on jaseg.de</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language><atom:link href="https://blog.jaseg.de/categories/index.xml" rel="self" type="application/rss+xml" />
</channel>
</rss>

4
config.toml
View file

@ -1,4 +0,0 @@
baseURL = "https://blog.jaseg.de/"
languageCode = "en-us"
title = "jaseg.de"
theme = "hugo-classic"

45
content/about.rst
View file

@ -1,45 +0,0 @@
---
title: "About jaseg"
---
About
-----
Hej, I'm Jan, or jaseg. At the moment I'm doing a PhD (Dr.-Ing.) at TU Darmstadt in Computer Science, specializing on
Hardware Security. This is my personal website where I publish things that I find interesting.
I self-host my code at `git.jaseg.de <https://git.jaseg.de/>`__, but I am also on `github <https://github.com/jaseg>`__
and on `gitlab <https://gitlab.com/neinseg>`__. I use github for issue tracking for some of my projects such as
`gerbolyze <https://github.com/jaseg/gerbolyze>`__ and `python-mpv <https://github.com/jaseg/python-mpv>`__. I maintain
the `python-mpv <https://pypi.org/project/python-mpv/>`__ and `gerbolyze <https://pypi.org/project/gerbolyze/>`__ python
packages on PyPI. Release tags on these two repositories are signed with the release signing key found `on github
<https://github.com/jaseg.gpg>`__ and below.
I am not on any social network, but feel free to write me an email at `hello@jaseg.de
<mailto:hello@jaseg.de?subject=About\ page\ on\ blog.jaseg.de>`__.
I do not use application-level email encryption such as S/MIME or PGP. If you need a higher level of secrecy than
regular old email provides, please ask around for my signal contact or email me a file encrypted using `age
<https://github.com/FiloSottile/age>`__ with one of the SSH keys listed `on my github
<https://github.com/jaseg.keys>`__. You can find both PGP and other SSH keys that I have used in the past on the
internet, but please consider these keys revoked, and do not use them to encrypt anything you send me.
Python package release signing key
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I use this GPG key (key ID ``ED7A208EEEC76F2D``) to sign git release tags of both `gerbolyze <https://github.com/jaseg/gerbolyze>`__ and `python-mpv
<https://github.com/jaseg/python-mpv>`__::
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEXom49xYJKwYBBAHaRw8BAQdA/KrWMt2MKGIZUvlQZnWjNd6i8/ZYjRsBQqEf
PJ8pJ+20NHB5dGhvbi1tcHYgUmVsZWFzZSBTaWduaW5nIEtleSA8cHl0aG9uLW1w
dkBqYXNlZy5kZT6IlgQTFggAPhYhBONvdTB/Cg7C0UX/XO16II7ux28tBQJeibj3
AhsDBQkSzAMABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEO16II7ux28thRYA
/3Yl1RdeUGor6K0RTxce9TIBB+DpLNupJgB9f6onuocpAQC614zQ/RQ6rkGTHCwA
ElFClWRQ5eppj0jpAuH15udqAbg4BF6JuPcSCisGAQQBl1UBBQEBB0A0mrXSv6rj
ajCmZR4H4OtowAx477YS+yWARqo1NtdgJwMBCAeIfgQYFggAJhYhBONvdTB/Cg7C
0UX/XO16II7ux28tBQJeibj3AhsMBQkSzAMAAAoJEO16II7ux28tMZwBAIUpHHvP
gRW2jQuzdw1r06kItfFk/0t+mgNUQ2+vtbhzAP98BoWx7lv+bvlIbBaVgLldusj0
pHnZI/0y3ksMBkdbBw==
=Mr6G
-----END PGP PUBLIC KEY BLOCK-----

102
content/imprint.rst
View file

@ -1,102 +0,0 @@
---
title: "Impressum"
---
Impressum
---------
.. raw:: html
<p>
Sebastian Götte<br/>
c/o Praxis Dr. Götte<br/>
Krankenhausstr. 1a<br/>
54634 Bitburg<br/>
imprint@jaseg.net
</p>
Inhaltlich Verantwortlicher gem. § 55 II RStV: Sebastian Götte (Anschrift s.o.)
Lizenz dieser Seite
-------------------
.. raw:: html
Dieses Werk ist lizenziert unter einer
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">
Creative Commons Namensnennung - Nicht-kommerziell - Weitergabe unter gleichen Bedingungen 4.0 International
Lizenz (CC-BY-NC-SA)
</a>.<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">
<img alt="Creative Commons Lizenzvertrag" style="border-width:0"
src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" />
</a>
Haftungsbeschränkung für Inhalte der Website
--------------------------------------------
Gemäß § 7 Abs. 1 TMG ist der Verantwortliche der Website i. S. v. § 5 TMG für eigene Informationen, die er zur Nutzung
bereithält, nach den allgemeinen Gesetzen verantwortlich.
Der Verantwortliche der vorbezeichneten Website übernimmt keine Haftung auf Aktualität, Richtigkeit und Vollständigkeit
der auf dieser Website zur Verfügung gestellten Inhalte. Dies gilt nicht, wenn dem Verantwortlichen vorsätzliches oder
grob fahrlässiges Verhalten vorzuwerfen ist. Die Inhalte wurden mit der größtmöglichen Sorgfalt erstellt. Dennoch kann
die inhaltliche Richtigkeit insbesondere bei komplexen Themen nicht gewährleistet werden, so dass der Verantwortliche
den Nutzern empfiehlt, bei wichtigen Informationen bei den zuständigen Stellen anzufragen oder rechtliche Beratung in
Anspruch zu nehmen. Sofern kostenpflichtige Inhalte oder Dienste auf der Website zur Verfügung gestellt werden, handelt
es sich dabei um unverbindliche Invitatio ad offerendum, welche lediglich zur Abgabe eines Angebots durch den Nutzer
aufrufen und selbst kein verbindliches Angebot darstellen.
Gemäß §§ 8 ff. TMG ist der Website-Betreiber für fremde Inhalte, die er für einen Nutzer veröffentlicht, nicht
verantwortlich, sofern
* er keine Kenntnis von der rechtswidrigen Handlung oder der Information hat und ihm im Falle von
Schadensersatzansprüchen auch keine Tatsachen oder Umstände bekannt sind, aus denen die rechtswidrige Handlung oder
die Information offensichtlich wird, oder
* er unverzüglich tätig geworden ist, um die Information zu entfernen oder den Zugang zu ihr zu sperren, sobald er
diese Kenntnis erlangt hat.
Erlangt der Website-Betreiber Kenntnis von solchen rechtswidrigen Inhalten, werden diese unverzüglich entfernt.
Haftung für ausgehende Links
----------------------------
Der Website-Betreiber verlinkt von seiner Website auf fremde Websites. Durch diese sog. „Hyperlinks“ wird der Nutzer
direkt auf die fremde Website geleitet. Der Website-Betreiber hat dabei keinerlei Einfluss auf die Informationen der
fremden Website. Daher kann keine Haftung für die Aktualität, Richtigkeit und Vollständigkeit der Inhalte der fremden
Website übernommen werden. Der Website-Betreiber versichert jedoch, dass ihm zum Zeitpunkt des Setzens der Verlinkung
keinerlei rechtliche Verstöße bekannt waren und er die fremde Website im Rahmen des Zumutbaren geprüft hat.
Erhält der Website-Betreiber Kenntnis von der Rechtswidrigkeit der verlinkten Inhalte, wird der entsprechende Link
unverzüglich entfernt.
Urheberrechte
-------------
Die auf dieser Webseite veröffentlichten Inhalte unterliegen dem deutschen Urheberrecht.
Als Urheber i. S. v. § 7 UrhG stehen dem Website-Betreiber die alleinigen und ausschließlichen Verwertungsrechte gemäß
§§ 15 ff. UrhG zu. Eine Vervielfältigung oder Verwendung sämtlicher Inhalte der Website in fremden elektronischen oder
gedruckten Medien ist ohne vorherige Zustimmung des Website-Betreibers untersagt und wird ggf. verfolgt.
Datenschutz
-----------
Die Website kann grundsätzlich ohne Eingabe von personenbezogenen Daten wie z. B. Name oder E-Mail-Adresse genutzt
werden. Sollte die Möglichkeit der Eingabe solcher Daten bestehen, so erfolgt die Mitteilung dieser Daten auf
freiwilliger Basis durch den Nutzer. Eine Weitergabe dieser Daten an Dritte ist ohne die ausdrückliche Zustimmung durch
den Nutzer ausgeschlossen.
Ein lückenloser Schutz der übermittelten Daten vor dem Zugriff durch unbefugte Dritte ist jedoch nicht möglich, da es im
Bereich der Datenübermittlung im Internet zu Sicherheitslücken kommen kann. Der Website-Betreiber versucht jedoch, die
Gefahr des unbefugten Zugriffs durch geeignete Maßnahmen zu unterbinden und im Falle der Kenntnis von einer
Sicherheitslücke diese durch geeignete Maßnahmen zu schließen.
Der Nutzung der im Impressum veröffentlichten Kontaktdaten durch Dritte zur Übersendung von nicht ausdrücklich
gewünschter Werbung, insbesondere Spam-Mails, wird widersprochen und im Falle der Nichtbeachtung ggf. mit rechtlichen
Schritten verfolgt.
(Quelle: `BUSE HERZ GRUNST Rechtsanwälte <https://www.kanzlei-wirtschaftsrecht.berlin/aktuelles/kostenloser-muster-disclaimer>`__)

209
content/posts/hsm-basics/index.rst
View file

@ -1,209 +0,0 @@
---
title: "Hardware Security Module Basics"
date: 2019-05-17T15:29:20+08:00
---
Hardware Security Modules and Security Research and Cryptography
================================================================
On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of
Prof. Mori's security research working group at Waseda University. The motivation for this was that outside of low-level
hardware security people and people working in the financial industry HSMs are not thought about that often. In
particular most network or systems security people would not consider them an option. Also it could turn out to be
really interesting to think about what could be done with an HSM in conjunction with modern cryptography (instead of
just plain old RSA-OAEP and AES-CBC).
`Click here to download a PDF with the slides for this talk. <mori_semi_hsm_talk_web.pdf>`__
Ideas for research in HSMs
==========================
Preparing for this talk brought me back to some research ideas I've been working on for a while now. Since I'm not sure
I'll find the time to properly research this topic, I thought it would be great to write down some rought outlines first
for future reference.
The Problem with current HSM tech
---------------------------------
Currently, HSMs are only used in certain specific niche applications such as certificate authority key management and
financial transaction data handling. One key reason for this is that HSMs currently don't provide the affordances that
would be needed for them to be adopted more widely by the cryptographic and security engineering community. As far as I
can tell, the two core missing affordances are:
1. To be more widely adopted, HSMs must become less expensive. Currently, they go for several tens of thousands of Euro,
which puts them outside most budgets.
2. To be more widely adopted, HSMs must provide the standardized programming interfaces familiar to cryptographic
developers. Currently, every HSM vendor has their own custom cryptographic API and a developer will have to train on
one specific vendor's tooling. Furthermore, any documentation of these internals is kept secret behind NDAs. This
constitutes a high barrier to entry, decreasing adoption in particular with young developers accustomed to
open-source ecosystems.
Attacking cost of implementation
--------------------------------
The first issue can be addressed by simply creating a viable low-cost alternative. There is no fundamental technical
reason for the high cost of HSMs. This cost is instead due to manufacturers trying to recoup their expenses for R&D as
well as certification from the small volumes HSMs are sold in.
Compared to system integration and certification the pure R&D cost of HSM defense mechanisms themselves is not too high
in an academic context it should be feasible to develop a sort of HSM blueprint that can then be cheaply produced by
anyone in need. Since the application areas outlined here are far from the core business areas of the clients of
established HSM vendors this would most likely not be a realistic threat to any established vendor's business and a
co-existence of both should not pose any problems in the short term.
Benefits of an academic HSM standard
------------------------------------
Tackling the high cost of current HSM hardware with an open-source HSM blueprint would yield
several academic advantages beyond cost reduction.
1. An open-source blueprint could serve as an academic reference design to evaluate and compare other HSM designs
against. For instance this would not only allow quantifying the effectiveness of academic security measures but also
allow an evaluation of commercial HSMs.
2. An open-source blueprint could stimulate academic research in this academically very quiet albeit commercially
important area. This research would ultimately benefit everyone employing HSMs by raising security standards in the
field. Since HSMs are never solely relied upon for overal system security both defensive and offensive security
research would yield these benefits.
3. An open-source blueprint would encourage new people to get into the field and both apply HSMs to practical problems
as well as improve HSMs themselves. Currently, this is highly discouraged due to the strictly proprietary nature of
all available systems.
4. Finally, developing an open-source HSM blueprint might yield new findings in adjacent academic areas due to the
hightly multi-disciplinary nature of security research in general and HSM design in particular.
Scope of an academic HSM standard
---------------------------------
An academic HSM blueprint would need to be flexible so that researchers can adapt it to their particular problem. A
modular architecture would lend itself to this flexibility. Fundamentally, there would be three components to this
architecture. First, a **base** containing infrastructure such as the surveillance microcontroller, power supplies,
power supply filtering and hardware DPA countermeasures, and possibly a standardized mechanical and electrical
interface.
Next to the base, a system integrator would put their *payload*. The nature of this payload is intentionally kept
unspecified, and it might be anything from a cryptographic microcontroller to a small embedded system such as a
raspberry pi single board computer. Keeping the *payload* open like this achieves two benefits: It gives the HSM
blueprint's user *their* familiar tooling and the hardware *they* need, allowing fast adoption. Someone well-versed in
e.g. Javascript could literally implement their cryptography in Javascript, run it on an off-the-shelf raspberry pi and
just apply the HSM blueprint around it. In addition, keeping the *payload* open reduces the scope of what needs to be
implemented. Building a general SDK on top of something like a bare ARM SoC such as a TI OMAP or a Freescale/NXP IMX
would be a considerable additional burden, on top of the actual HSM design. Keeping the *payload* open allows research
to concentrate on the actual point, the HSM design.
The final and most important component would be a set of *security measures* that can be combined with the base to
form the final HSM. Each of these *security measures* would entail a detailed specification of its design, manufacture
and security properties. These *security measures* could be simple things like tamper switches or potting, but could
also be complex things like security meshes.
Given these three components -- *base*, *payload* and *security measures* as detailed specifications any engineer should
be able to design and manufacture a HSM customized to their needs. Unifying these three components within the HSM
blueprint would be a set of reference designs. Each reference design would implement a particular parametrization of the
three architectural components with a physical hole cut out where the payload would go.. These reference designs would
for one serve to guide any implementer on the customization and integration of their own derivation from the blueprint.
In addition it would serve as an extremely simple, low-cost point of entry into the ecosystem. A curious researcher
could simply replicate the reference design and put their existing payload inside. Practically this might mean e.g. a
researcher having PCBs produced according to the design files for a reference design for a mesh-based HSM, producing
their own mesh, physically glueing a raspberry pi SBC into the middle of it, and potting the resulting system. Given the
ease of prototype PCB fabrication today this would realistically allow evaluation of HSM technologies on a budget that
is orders of magnitude less than the cost of current HSMs.
Research ideas for tamper detection mechanisms
==============================================
The core component of an HSM blueprint would be a suite of tamper detection mechanisms. Following are a few ideas on how
to improve on the current state of the art of membrane tamper switches plus temperature sensors plus PCB and printed
security meshes plus potting.
Improvements on existing techniques
-----------------------------------
Light sensors
~~~~~~~~~~~~~
**Advanced analog sensing**
**Self-test functionality**
Security meshes
~~~~~~~~~~~~~~~
**Analog sensing**
DIY or small lab mesh production
--------------------------------
**3D metal patterning techniques** refers to any technique for producing thin, patterned metal structures on a
three-dimensional plastic substrate. The basic process would consist of 3D-printing the polymer substrate, depositing a
thin metal layer on top and then patterning this metal layer. A good starting point here would be the recent work of
`Ben Kraznow`_ on this exact thing.
**Copper filament methods** would be any method embedding copper wire from a spool in some resin or other matrix. This
could mean either of a systematic approach of carefully winding or folding the copper wire into patterns or a
non-systematic approach of simply stuffing a large tangle of copper wire into a small space. The main challenge with the
former would be to find a non-tedious way of production. The main challenge with the latter would be to find process
parameters that guarantee complete coverage of the HSM without holes or other areas of lower sensitivity to intrusions.
Both approaches would require careful consideration of the overall design including the polymer resin supporting
structure to ensure sensitivity against attacks since copper wire is mechanically much stronger than the micrometre-thin
metal coatings used in patterning techniques.
Envelope measurement
--------------------
Finally, I think there is another set of currently under-utilized tamper-detection methods that would be very
interesting to explore. I am not aware of an academic term for these, so I am just going to dub them *envelope
measurement* here.
The fundamental apporach of a mesh is to build a physical security envelope (the mesh) that physically detects when it
is disturbed (open or short circuits). This approach works well but has the disadvantage that these meshes are rather
complex to manufacture since effectively every part of them is acting as a sensing element. A conceptually more complex
but in practice potentially simpler approach might be to split the functions of security envelope and sensing element.
This would mean that in place of the mesh, some form of passive element such as metal foil forms the security envelope
which is then checked for tampering using a very sensitive sensor inside. This remote-sensing approach might simplify
the manufacture of the envelope itself and thus yield a design that is more easily customized. Following are a few ideas
on how to approach this envelope measurement problem.
**Ultrasonic** If the HSM is potted, a few ultrasonic transducers could be added inside the potting. With several
transducers, any one could be used to transmit ultrasound while the others measure complex phase and energy of the
signal they receive. The circuitry for this could be made fairly simple if using a static transmit frequency or a low
chirp rate by using a homodyne receiver built around a comparator fed into some timers. This approach would likely
detect any mechanical attack and would also rule out chemical attacks involving liquids (though starting from which
amount of liquid depends on receiver sensitivity). The main disadvantages might be high power consumption and cost and
size of the ultrasonic transducers. Traditional cheap transducers made for air as a transmission medium are fairly large
and might not adequately couple into potting compound. If somehow one could convince a standard small piezo element to
do the same job that would be great as far as cost and size are concerned. A concern in some fringe use cases might be
suceptibility to ambient noise, though this could easily be reduced at the expense of space and heat dissipation
capacity by adding sound dampening on the outside. A likely attack vector against this approach might be using a laser
cutter to drill a hole through the potting compound, then inserting probes carefully chosen to not couple too much
to the potting compound ultrasonically.
**Light** In either an unpotted HSM or one potted with a transparent (at some wavelengths) potting compound one could
embed LEDs and photodiodes in a similar setup to the ultrasonic setup described above. In contrast to the ultrasound,
the LEDs would literally have to light up the HSM's interior and shadows might be an issue since the HSM is likely some
flat rectangular shape. A possible solution to this would be to coat both the embedded payload and the lid with some
highly reflective paint such as some glossy silver paint or simple white paint. The basic approach might be as simple as
simply turning on several LEDs distributed throughout the HSM in turn and measuring amplitude at several photodetectors,
or as complex as doing a LIDAR-like phase measurement sweeping through a range of frequencies to determine not only
absorption but also phase/distance characteristics between emitter LED and detector photodiode. Using some high-gain TIA
along with a homodyne detector (lock-in amplifier) and changing emitter intensity, very precise measurements of both
absorption and phase might be possible, as might be measurements through almost opaque, diffuse potting compounds such
as a grey epoxide resin. The main disadvantages of this method would likely be the need to thoroughly light-proof the
entire HSM (likely by wrapping it in metal foil) and the potentially high cost of transmitter and receiver circuitry
(nice TIAs aren't cheap). To be effective against attacks using e.g. very fine drills and probes the system would likely
have to be very sensitive.
**Radar** Finally, one could turn to standard radar techniques to fingerprint the inside of the HSM. The goal here would
be fingerprinting instead of mapping since only changes need to be detected. In this approach one could use homodyne
detection to improve sensitivity and reduce receiver complexity, and sweep frequencies similar to an FMCW radar (but
probably without exploiting the self-demodulation effect). Besides high cost, this approach has two disadvantages.
First, such a system would likely not go beyond 24GHz or maybe 40GHz due to component availability issues. Even at 40GHz
the wavelength in the potting compound would be in the order of magnitude of several millimeters. Fine intrusions using
some tool chosen to not interact too much with the EM field inside the HSM such as a heated ceramic needle or simply a
laser cutter might not be detectable using this approach. In any case, this system would certainly not be able to detect
small holes piercing the HSM enclosure. The HSM enclosure would have to be made into an RF shield, likely by using some
metal foil in it.
Overall in the author's opinion these three techniques are most promising in order *Light*, *Ultrasonic*, *Radar*. Light
would prbably provide the best sensitivity at expense of some cost. Ultrasonic might be used in conjunction with light
to cover some additional angles since it is potentially very low-cost. Radar seems hard to engineer into a solution that
works reliably and also would likely be at least an order of magnitude more expensive than the other two technologies
while not providing better sensitivity.
.. _`Ben Kraznow`: https://www.youtube.com/watch?v=Z228xymQYho
.. _affordances: https://en.wikipedia.org/wiki/Affordance

221
content/posts/kicad-mesh-plugin/index.rst
View file

@ -1,221 +0,0 @@
---
title: "Kicad Mesh Plugin"
date: 2020-08-18T13:15:39+02:00
---
.. raw:: html
<figure>
<img src="images/anim.webp" style="max-width: 20em">
</figure>
Tamper Detection Meshes
=======================
Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and
mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious
surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for
groceries with a credit card, messaging a friend on `Signal <https://signal.org>`_ or unlocking a car with its keyfob
would not be possible. The security of all of these systems in its core lies on the secrecy of cryptographic keys.
Systems differ in what kind of keys they use, how often these keys are replaced and the intricacies of the cryptographic
operations these keys fit into but all have in common that their security relies on keeping the keys secret.
In practice, this secrecy has been implemented in many different ways. Mass-market software such as browsers or
messenger apps usually relies on some operating system facility to tell the computer "*please keep this piece of memory
away from all other applications*". While on desktop operating systems usually this does not provide much of a barrier
to other programs on the same computer, on modern mobile operating systems this approach is actually quite secure.
However, given sufficient resources no security is perfect. All of these systems can be compromised if the host
operating system is compromised sufficiently, and for organizations with considerable resources a market has sprung up
that offers turn-key solutions for all wiretapping needs.
In some applications, this level of security has not been considered sufficient. Particularly financial infrastructure
is such a high-profile target that a lot of effort has been put into the security of cryptographic implementations. The
best cryptographic algorithm is useless if it is run on a compromised system (from that system's point of view anyway).
One of the core cryptographic components in financial applications are smartcards like they are used as payment cards in
most countries nowadays. These smartcards contain a small, specialized cryptographic microcontroller that is designed to
be hard to tamper with. Though one of the design goals of the system is to reduce the amount of sensitive information
stored on the card, things such as copying of a card can only be hindered by making the chip hard to read out.
.. raw:: html
<figure>
<img src="images/modern_art.svg" style="max-width: 20em">
</figure>
With smartcards being the means of choice on one side of the counter in electronic payments, on the other side of the
counter a different technology prevails. Attacks on payment terminals are bound to have much more dire consequences than
attacks on individual cards since one terminal might see hundreds of cards being read every day. For this reason, the
level of attack countermeasures employed in these terminals is a considerable step up from bare smartcards. While a
smartcard is made physically hard to tamper, it does not have a battery and it can only detect tampering once it is
powered by a reader. This allows for well-equipped attackers to use tools such as Focused Ion Beam (FIB) workstations to
circumvent the smartcard's defences while it is powered down, and then power up the card to carry out the actual attack.
The answer to this problem in electronic payment infrastructure is called *Hardware Security Module*, or HSM. An HSM is
similar to a smartcard in its function (cryptographic processing using keys that are meant to never leave the protection
of the HSM). The one major between the two is that an HSM has its own battery and is continuously powered from its
manufacture to the day it is scrapped. If the HSM looses power at any point in time, it uses a small amount of energy
stored internally to securely wipe all cryptographic secrets from its memory within a few milliseconds.
Being powered at all times allows the HSM to actively detect and respond to attacks. The most common way this is done is
by wrapping the juicy secret parts in a foil or a printed circuit board that is patterned with a long and convoluted
maze of wires, called a *mesh*. The HSM is continuously monitoring these wires for changes (such as shorts, breaks or
changes in resistance) and will sound the alarm when any are detected. Practically, this presents a considerable hurdle
to any attacker: They have to find a way to disable or circumvent the mesh while it is being monitored by the HSM. In
practice, often this is no insurmountable challenge but it again increases attack costs.
DIY Meshes
==========
Throughout my studies in security research I have always had an interest in HSMs. I have taken apart my fair share of
HSMs and at this point, to understand the technology more, I want to experiment with building my own HSM. In last year's
`HSM basics <{{<ref "posts/hsm-basics/index.rst">}}>`_ post I have lined out some ideas for a next generation design that
deviates from the bread-and-butter apporoach of using a mesh as the primary security feature. Before embarking on
practical experiments with these ideas, I want to first take a stab at replicating the current state of the art as best
I can. State of the art meshes often use exotic substrates such as 3D plastic parts with traces chemically deposited on
their surface or special flexible substrates with conductive ink traces. These technologies will likely be too
cumbersome for me to implement myself only for a few prototypes, and industrial manufacturers will most likely be too
expensive. Thus, I will concentrate on regular PCB technology for now.
The idea of a mesh on a PCB is pretty simple: You have one or several traces that you try to cover every corner of the
mesh PCB's area with. To be most effective, the traces should be as thin and as close together as possible. To increase
the chances of a manipulation being detected, multiple traces can also be used that can then be monitored for shorts
between them.
While one can feasibly lay out these traces by hand, this really is an ideal application of a simple auto-router. While
general PCB autorouting is *hard*, auto-routing just a few traces to approximate a space-filling curve is not. Since I
am just starting out, I went with the simplest algorithmic solution I could think of. I first approximate the area
designated to the mesh with a square grid whose cells are a multiple of my trace/space size. The mesh will only be drawn
into grid cells that are fully inside the set boundaries. All cells outside or going across the border are discarded in
this step.
I decided to implement this auto-router in a KiCAD plugin. Though KiCADs plugin API is not the best, it was just about
usable for this task.
.. raw:: html
<figure>
<img src="images/kicad-mesh-outline.png" alt="KiCAD showing an irregular board shape with rounded corners and
indents. In the middle of the board there is a footprint for a 4-pin surface-mount pin header.">
<figcaption>The process starts out with the mesh shape being defined inside KiCAD. The mesh's outline is drawn
onto one of the graphical "Eco" layers. A footprint is placed to serve as a placeholder for the mesh's
connections to the outside world. This footprint is later used as the starting point for the mesh generation
algorithm.</figcaption>
</figure>
.. raw:: html
<figure>
<img src="images/grid-vis-plain.svg" alt="A vizualization of the grid fitting process. Over the mesh's irregular
outline a grid is drawn. In this picture, all grid cells that are fully inside the grid are shown. Grid cells
that overlap the mesh border are highlighted. Grid cells outside of the mesh border are not drawn.">
<figcaption>A visualization of the grid fitting process. First, a grid large enough to contain the mesh border
is generated. Then, every cell is checked for overlap with the mesh border area. If the cell is fully inside, it
(yellow), it is considered in the mesh generation later. Cells outside (gray) or on the border (red) are
discarded.</figcaption>
</figure>
After generating the grid, starting from the place I want to connect to the mesh, I walk the grid's cells one by one to
generate a tree that covers the entire grid's area. To set the mesh's starting place I place a footprint on the board
(dark gray in the picture above) whose designator I then tell my script. The tree generation algorithm looks like a
depth-first search, except all checks are random. Depending on the level of randomness used at each step of the
algorithm it yields more or less organized-looking results. Below are five example runs of the algorithm at differing
levels of randomness with the cells colored according to their distance from the tree root. 0% randomness means that the
algorithm is going to try cells in forward direction first on every step, and only then try out left and right. 100%
means that on every step, the algorithm is choosing a new direction at random.
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/cells-0.svg" alt="a completely organized looking grid with spiral patterns all over.">
<figcaption>0%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-25.svg">
<figcaption>25%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-50.svg">
<figcaption>50%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-75.svg">
<figcaption>75%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-100.svg" alt="a completely random looking grid with cells aggregating into ireggular
areas that look like paint splotches.">
<figcaption>100%</figcaption>
</figure>
</figure>
After I have built this tree like you would do in a depth-first search, I draw my one or several mesh mesh traces into
it. The core observation here is that there is only 16 possible ways a cell can be connected: It has four neighbors,
each of which it can either be connected to or not, which results in 2^4 options. If you consider rotations and
mirroring, this works out to rotations or mirrored versions of only six base tiles: The empty tile, a tile with all four
sides connected, a straight through, a 90 degree bend, and a "T"-junction—see the illustration below.
.. raw:: html
<figure>
<img src="images/maze_tiles_plain.svg" style="max-width: 20em">
<figcaption>
There are six possible tile types in our connectivity graph inside its square tiling. This graphic illustrates
all sixteen rotations of these with how they would look in a two-conductor mesh.
</figcaption>
</figure>
After tiling the grid according to the key above, we get the result below.
.. raw:: html
<figure>
<img src="images/tiles-25-small.svg">
<figcaption>
An auto-routed mesh with traces colored according to tile types.
</figcaption>
</figure>
.. raw:: html
<figure>
<img src="images/traces-25-small.svg">
<figcaption>
The same mesh, but with traces all black.
</figcaption>
</figure>
Putting it all together got me the KiCAD plugin you can see in the screenshot below.
.. raw:: html
<figure>
<img src="images/kicad-mesh-settings2.png">
<figcaption>
The plugin settings window open.
</figcaption>
</figure>
.. raw:: html
<figure>
<img src="images/kicad-mesh-result-large.png">
<figcaption>
After runing the plugin, the generated mesh looks like this in pcbnew.
</figcaption>
</figure>
I am fairly happy with the result, but getting there was a medium pain. Especially KiCAD's plugin API is still very
unfinieshed. It is hard to use, most parts are completely undocumented and if you use anything but its most basic parts
things tend to break. One particular pain point for me was that after generating the mesh, the traces have been added to
the board, but are still invisible for some reason. You have to save the board first, then re-load the file for them to
become visible. Also KiCAD crashes whenever the plugin tries to remove a trace, so currently my workflow involves always
making a copy of the board file first and treating mesh generation as a non-reversible finishing step.
`Check out the code on my cgit <https://git.jaseg.de/kimesh.git/tree/plugin/mesh_dialog.py>`_.
.. ::
.. raw:: html
<figure>
<img src="images/grid-vis-plain.svg" alt="">
<figcaption></figcaption>
</figure>

500
content/posts/led-characterization/index.rst
View file

@ -1,500 +0,0 @@
---
title: "LED Characterization"
date: 2018-05-02T11:18:38+02:00
---
Preface
-------
Recently, I have been working on a `small driver`_ for ambient lighting using 12V LED strips like you can get
inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up
to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when
fading between different colors, the colors look *all wrong*! This observation led me down a rabbit hole of color
perception and LED peculiarities.
The idea of the LED driver was that it can be used either with up to eight single-color LED tapes or, much more
interesting, with up to two RGB or RGBW (red-green-blue-white) LED tapes. For ambient lighting high color resolution was
really important so you could dim it down a lot without flickering. I ended up using the same driver stage I used in the
`multichannel LED driver`_ project for its great color resolution and low hardware requirements.
.. raw:: html
<figure>
<img src="images/rgb_cube.svg" alt="An illustration of the RGB color cube.">
<figcaption>An illustration of the RGB color cube.
<a href="https://commons.wikimedia.org/wiki/File:RGB_color_cube.svg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Maklaan">Maklaan from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure>
To make setting colors over Wifi more intuitive I implemented support for HSV colors. RGB is fine for communication
between computers, but I think HSV is easier to work with when manually inputting colors from the command line. RGB is
close to how most monitors, cameras and the human visual apparatus work on a very low level but doesn't match
higher-level human color perception very well. When we describe a color we tend to think in terms of "hue" or
"brightness", and computing a measure of those from RGB values is not easy.
Colors and Color Spaces
-----------------------
`Color spaces`_ are a mathematical abstraction of the concept of color. When we say "RGB", most of the time we actually
mean `sRGB`_, a standardized notion of how to map three numbers labelled "red", "green" and "blue" onto a perceived
color. `HSV`_ is an early attempt to more closely align these numbers with our perception. After HSV, a number of other
*perceptual* color spaces such as `XYZ (CIE 1931)`_ and `CIE Lab/LCh`_ were born, further improving this alignment. In
this mathematical model, mapping a color from one color space into another color space is just a coordinate
transformation.
.. raw:: html
<figure>
<img src="images/hsv_cylinder.png" alt="An illustration of the HSV color space as a cylinder.">
<figcaption>An illustration of the HSV color space as a cylinder.
<a href="https://commons.wikimedia.org/wiki/File:HSV_color_solid_cylinder.png">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:SharkD">SharkD from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure>
CIE 1931 XYZ is much larger than any other color space, which is why it is a good basis to express other color spaces
in. In XYZ there are many coordinates that are outside of what the human eye can perceive. Below is an illustration of
the sRGB space within XYZ. The wireframe cube is (0,0,0) to (1,1,1) in XYZ. The colorful object in the middle is what
of sRGB fits inside XYZ, and the lines extending out from it indicate the space that can be expressed in sRGB but not in
XYZ. The fat white curve is a projection of the *monochromatic spectral locus*, that is the curve of points you get in
XYZ for pure visible wavelengths.
As you can see, sRGB is *much* smaller than XYZ or even the part within the monochromatic locus that we can perceive. In
particular in the blues and greens we loose *a lot* of colors to sRGB.
.. raw:: html
<figure>
<video controls loop>
<source src="video/sRGB.mkv" type="video/h264">
<source src="video/sRGB.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured sRGB color space within XYZ. The thick, white line is the spectral
locus.
<a href="video/sRGB.mkv">mkv/h264 download</a> /
<a href="video/sRGB.webm">webm download</a>
</figcaption>
</figure>
The wrong colors I got when fading between colors were caused by this coordinate transformation being askew. Thinking
over the problem, there are several sources for imperfections:
* The LED driver may not be entirely linear. For most modulations such as PWM the brightness will be linear starting
from a certain value, but there is probably an offset caused by imperfect edges of the LED current. This offset can be
compensated with software calibration. I built a calibration setup for driver linearity in the `multichannel LED
driver`_ project. Below are pictures of ringing on the edges of an LED driver's waveform.
* The red, green and blue channels of the LEDs used on the LED tape are not matched. This skews the RGB color space.
In practice, the blue channel of my RGB tape to me *looks* much brighter than the red channel.
* The precise colors of the red, green and blue channels of the LEDs are unknown. Though the red channel *looks* red, it
may be of a slightly different hue compared to the reference red used in `sRGB`_ which would also skew the RGB color
space.
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/driver_ringing_strong.jpg" alt="Strong ringing on the LED voltage waveform edge at about
100% overshoot during about 70% of the cycle time.">
<figcaption>The LED strip being at the end of a couple meters of wire caused extremely bad ringing at high
driving frequencies.</figcaption>
</figure><figure class="side-by-side">
<img src="images/driver_ringing_weak.jpg" alt="Weak ringing on the LED voltage waveform edge at about 30%
overshoot during about 20% of the cycle time.">
<figcaption>Adding a resistor in front of the MOSFET gate to slow the transition dampened the ringing
somewhat, but ultimately it cannot be eliminated entirely.</figcaption>
</figure>
</figure>
These last two errors are tricky to compensate. What I needed for that was basically a model of the *perceived* colors
of the LED tape's color channels. A way of doing his is to record the spectra of all color channels and then evaluate
their respective XYZ coordinates. If all three channels are measured in one go with the same setup the relative
magnitudes of the channels in XYZ will be accurate.
To map any color to the LEDs, the color's XYZ coordinates simply have to be mapped onto the linear coordinate system
produced by these three points within XYZ. LEDs are mostly linear in their luminous flux vs. current characteristic so
this model will be adequate. The spectral integrals mapping the channels' measured responses to XYZ need only be
calculated once and their results can be used as scaling factors thereafter.
Measuring the spectrum
----------------------
In order to compensate for the cheap LED tape's non-ideal performance I had to measure the LED's red, green and blue
channels' spectra. The obvious thing would be to go out and buy a `spectrograph`_, or ask someone to borrow theirs. The
former is kind of expensive, and I did not want to wait two weeks for the thing to arrive. The latter I could probably
not do every time I got new LED tape. Thus the only choice was to build my own.
Luckily, building your own spectrometer is really easy. The first thing you need is something that splits incident light
into its constituent wavelengths. In professional devices this is called the *`monochromator`_*, since it allows extraction
of small color bands from the spectrum. The second thing is some sort of optics that project the incident light onto a
screen behind the monochromator. In professional devices lenses or curved mirrors are used. In a simple homebrew job a
pinhole as you would use in a `camera obscura`_ does a remarkably nice job.
For the monochromator component several things could be used. A prism would work, but I did not have any. The
alternative is a `diffraction grating`_. Professional gratings are quite specialized pieces of equipment and thus
rather expensive. Luckily, there is a common household item that works almost as well: A regular CD or DVD. The
microscopic grooves that are used to record data in a CD or DVD work the same as the grooves in a professional
diffraction grating.
Household spectra
-----------------
From this starting point, a few seconds on my favorite search engine yielded an `article by two researchers from the
National Science Museum in Tokyo`_ providing a nice blueprint for a simple cardboard-and-DVD construction for use in
classrooms. I replicated their device using a DVD and it worked beautifully. Daylight and several types of small LEDs I
had around did show the expected spectra. Small red, yellow, green, and blue LEDs showed narrow spectra, daylight one
continuous broad one, and white LEDs a continuous broad one with a distinct bright spot in the blue part. The
single-color LED spectra are quite narrow since they are determined by the LED's semiconductor's band gap, which is
specific to the semiconductor used and is quite precise. White LEDs are in fact a blue LED chip covered with a so-called
*phosphor*. This phosphor is not elementary phosphorus but an anorganic compound that absorbs the LED chip's blue light
and re-emits a broader spectrum of more yellow-ish wavelengths instead. The final LED spectrum is a superposition of
both spectra, with some of the original blue light leaking through the phosphor mixing with the broadband yellow
spectrum of the phosphor.
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/spectrograph_step1_parts.jpg">
<figcaption>The ingredients. The cup of coffee and Madoka Magica DVD set are essential to the eventual
function of the appartus.</figcaption>
</figure><figure class="side-by-side">
<img src="images/spectrograph_step2.jpg">
<figcaption>Step 1: Cut to size and mark down all holes as described in <a
href="http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf">the manual</a></figcaption>
</figure>
<figure class="side-by-side">
<img src="images/spectrograph_step3.jpg">
<figcaption>Step 2: Cut out all holes</figcaption>
</figure><figure class="side-by-side">
<img src="images/spectrograph_step4_complete.jpg">
<figcaption>The finished result with the back side showing. The viewing window is on the bottom of the other
side.</figcaption>
</figure>
</figure>
Now that I had a spectrograph, I needed a somewhat predictable way of measuring the spectrum it gave me.
Measuring a spectrum
--------------------
Pointing a camera at the spectrograph would be the obvious thing to do. This produces pretty images but has one critical
flaw: I wanted to acquire quantitative measurements of brightness across the spectrum. Since I don't have a precise
technical datasheet specifying the spectral response of any of my cameras I can't compare the absolute brightness of
different colors on their pictures. Some other sensor was needed.
.. raw:: html
<figure>
<img src="images/daylight_spectrum_dvd.jpg">
<figcaption>The daylight spectrum as seen using a DVD as a grating.
<a href="https://commons.wikimedia.org/wiki/File:SpectresSolaires-DVD.jpg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Xofc">Xofc from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA 4.0</a>
</figcaption>
</figure>
Measuring light intensity
~~~~~~~~~~~~~~~~~~~~~~~~~
Looking around my lab, I found a bag of `SFH2701`_ visible-light photodiodes. Their
datasheet includes their spectral response so I can compensate for that, allowing precise-ish absolute intensity
measurements. Just like LEDs, photodiodes are extremely linear across several orders of magnitude. The datasheet of the
classic `BPW34`_ photodiode shows that this photodiode's light current is exactly proportional to illuminance over at
least three orders of magnitude. The `SFH2701`_ datasheet does not include a similar graph but its performance will be
similar. The `SFH2701`_ photodiodes I had at hand were perfect for the job compared to the vintage `BPW34`_ since their
active sensing area is really small (0.6mm by 0.6mm) compared to the BPW34 (a whopping 3mm by 3mm). If I were to use a
`BPW34`_ I would have to insert some small apterture in front of it so it does not catch too broad a part of the
spectrum at once. The `SFH2701`_ is small enough that if I just point it at the projected spectrum directly I will
already get only a small part of the spectrum inside its 0.6mm active area.
To convert the photodiode's tiny photocurrent into a measurable voltage I built another copy of the `transimpedance
amplifier`_ circuit I already used in the `multichannel LED driver`_. A `transimpedance amplifier`_ is an
amplifiert that produces a large voltage from a small current. The weird name comes from the fact that it works kind of
like an amplified resistor (which can be generalized as an *impedance* electrically). Apply a current to a resistor and
you get a voltage. A transimpedance amplifiert does the same with the difference that its input always stays at 0V,
making it look like an ideal current sink to the connected current source.
Transimpedance amplifiers are common in optoelectronics to convert small photocurrents to voltages. In this instance I
built a very simple circuit with a dampened transimpedance amplifier stage followed by a simple RC filter for noise
rejection and a regular non-inverting amplifier using another op-amp from the same chip to further boost the filtered
transimpedance amplifier output. I put all the passives setting amplifier response (the gain-setting resistors and the
filter resistor and capacitors) on a small removable adapter so I could easily change them if necessary. I put a small
trimpot on the virtual ground both amplifers use as a reference so I could trim that if necessary.
.. raw:: html
<figure>
<img src="images/preamp_schematic.jpg" alt="A drawing of the photodiode preamplifier's schematic">
<figcaption>The photodiode preamplifier schematic. Schematic drawn with an unlicensed copy of
DaveCAD.</figcaption>
</figure>
Following are pictures of the preamplifier board. The connectors on the top-left side are two copies of the analog
signal for the ADC and a small panel meter. The SMA connector is used as the photodiode input since coax cables are
generally low-leakage and have built-in shielding. The circuit is powered via the micro-USB connector and the analog
ground bias voltage can be adjusted using the trimpot.
For easy replacement, all passives setting gain and frequency response are on a small, pluggable carrier PCB made from a
SMD-to-DIP adapter.
Flying-wire construction is just fine for this low-frequency circuit. In a high-speed photodiode preamp, the
transimpedance amplifier circuit would be highly sensitive to stray capacitance, but we're not aiming at high speed
here.
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/preamp_front.jpg">
<figcaption>The front side of the preamplifier board.</figcaption>
</figure><figure class="side-by-side">
<img src="images/preamp_back.jpg">
<figcaption>The wiring of the photodiode preamp.</figcaption>
</figure>
</figure>
Given a way to measure intensity what remains missing is a way to scan a single photodiode across the spectrum.
Scanning the projection
~~~~~~~~~~~~~~~~~~~~~~~
A cheap linear stage can be found in any old CD or DVD drive. These drives use a small linear stage based on a
stepper-driven screw to move the laser unit radially. Removing the laser unit and connecting a leftover stepper driver
module I was left with a small linear stage with about 45 steps per cm without microstepping enabled. The driver I used
was an `A4988`_ module that required at least 8V motor drive voltage. I used a small micro USB-input boost converter
module to generate a stable 10V supply for the motor driver, with the USB's 5V rail used as a logic supply for the motor
driver.
The `SFH2701`_ can easily be mounted to the linear stage using a small SMD breakout board glued in place with thin wires
connecting it to the transimpedance amplifier. The DVD drive linear stage is not very strong so it is important that
this wire does not put too much strain on it.
Above the photodiode, I mounted a small piece of paper on the linear stage to be used as a projection screen to align
the linear stage in front of the spectrometer viewing window. A line on the screen paper points to the photodiode die in
parallel to the linear stage allowing precise alignment.
The whole unit with photodiode preamplifier, linear stage, photodiode and stepper motor driver finally looks like this:
.. raw:: html
<figure>
<img src="images/electronics_whole.jpg" alt="The complete electronics setup of the spectrograph. In the back
there is the DVD drive stepper stage. In front of it, mounted on a piece of wood are a small USB-to-12V
switching-regulator module to power the stepper motor in the top left, below on the bottom left is the
photodiode preamp and on the right is a breadboard with the stepper driver module and lots of jumper wires
interconnecting everything. On the right of the breadboard, a buspirate is attached to interface everything to a
computer. On the bottom edge of the piece of wood, two LED panel meters are mounted for readout of the preamp
output and the stepper supply voltages.">
<figcaption>The complete electronics setup. The buspirate on the right interfaces to a computer and controls the
stepper driver and ADC'es the preamp output. The two panel meters show the preamp output and stepper voltage for
setup.</figcaption>
</figure>
The projection of the spectrum can be adjusted by moving the light source relative to the entry slot and by moving
around the grating DVD.
The capture process
~~~~~~~~~~~~~~~~~~~
To capture a spectrum, first the light source has to be mounted near the spectrograph's entry slot. The LED tape I
tested I just taped face-down directly into it. Next, the grating DVD has to be adjusted to make sure the spectrum
covers a sensible part of the photodiode's path. Mostly, this boils down to adjusting the photodiode distance and height
to match the vertical extent and wiggling the grating DVD to adjust the projection's horizontal position.
After the optics are set-up, the photodiode preamplifier has to be adjusted. In my experiments, most LED tape at 5GΩ
required a high-ish amplification. The goal in this step is to maximize the peak response of the preamp to be just
shy of its VCC rail to make best use of its dynamic range. To adjust the pre-amp, I took several very coarsely-spaced
measurements to give me an estimate of the peak while I did not yet know its precise location.
Since stray daylight totally swamped out the weak projection of the LED's spectrum I shielded the entire setup with a
small box made of black cardboard and two black t-shirts on top. This shielding proved adequate for all my measurements
but I had to be careful not to accidentially move the DVD that was stuck into the spectrograph with the shielding
t-shirts.
For capturing a single spectrum I wrote a small python script that will automatically move the stepper in adjustable
intervals and take two measurements at each point, one with the LED tape off that can be used for offset calibration and
one with the LED tape on. All measurements are stored in a sqlite database that can then be accesssed from other
scripts.
I built a small script that shows the progress of the current run and an jupyter notebook for data analysis. The jupyter
notebook is capable of live-updating a graph with the in-progress spectrum's data. This was quite useful as a sanity
check for when I made some mistake easy to spot in the resulting data.
After one color channel is captured, the LED tape has to be manually set to the next color and the next measurement can
begin.
.. raw:: html
<figure>
<img src="images/raw_plot_cheap_rgb.svg" alt="A plot with three wide peaks, two large peaks on both sides and
one smaller one in the middle. The middle one overlaps the two on the sides. The large ones are about 2.5V in
amplitude. Overall, the plot is about 300 stepper steps wide with each peak being around 130 steps wide.">
<figcaption>A plot of the raw preamp output voltage versus stepper position. From left to right, the three peaks
are blue, green and red. Step 0 corresponds to the bottommost stepper position and the shortest wavelength.
</figcaption>
</figure>
Data analysis
~~~~~~~~~~~~~
Data analysis consists of three major steps: Offset- and stray light removal, wavelength and amplitude calibration and
color space mapping.
Offset removal
**************
The first task is to remove the offset caused by dark current as well as stray light of the LED's bright primary
reflection on the DVD. The LED is very bright and only a small part of its light gets reflected by the grating towards
the photodiode screen. The remaining part of the light is reflected onto the table in front of the DVD spectrograph.
Though I covered all of this with black cardboard, some of that light ultimately gets reflected onto the photodiode.
This causes a large offset, in particular in the blue part of the spectrum since in this part the photodiode is closest
to the spectrograph's opening.
The composite offset can be approximated with a second-order polynomial that is fitted to all the data outside of the
main peak's area. Since at this point the wavelength of each data point is still unknown this is done with a rough first
estimate of the three colors' peaks' locations and widths.
Wavelength- and amplitude calibration
*************************************
The photodiode's response is strongly wavelength-dependent. In particular in the blue band, the photodiode's sensitivity
gets very poor down to about 20% at the edge to ultraviolet. This effect is strong enough to move the apparent location
of the blue peak towards red.
.. raw:: html
<figure>
<img src="images/photodiode_sensitivity.svg" alt="A plot of photodiode sensitivity against wavelength relative
to peak sensitivity at 820nm. The sensitivity rises from 20% at 380nm approximately linearly to 80% at 620nm,
then the rise rolls off.">
<figcaption>A plot of the photodiode's relative sensitivity in the visible spectrum. The sensitivity is
normalized against its peak at 820nm.
</figcaption>
</figure>
The problem is that in order to remove this non-linearity, we would already have to know the wavelength of the measured
light. Since I don't, I settled for a two-step process. First, a coarse wavelength calibration is done relative to the
red peak and the short-wavelength edge of the blue peak. The photodiode measurements are then sensitivity-corrected
using this coarse measurement. Then all three channel peaks are measured in the resulting data and a fine wavelength
estimate is produced by a least-squares fit of a linear function. This fine estimate is then used for a second
sensitivity correction of all original measurements and the scale is changed from stepper motor step count to
wavelength in nanometers.
.. raw:: html
<figure>
<img src="images/processed_plot_cheap_rgb.svg" alt="A plot with three wide peaks, all three of different
heights. The leftmost peak is highest at 6nA, the middle peak lowest at 1.6nA and the rightmost peak in between
at 4nA. The middle one overlaps the two on the sides. Overall, the plot spans about 300nm on its x axis with
each peak being around 100nm wide.">
<figcaption>A plot of the processed measurements. From left to right, the three peaks are blue, green and red.
</figcaption>
</figure>
.. FIXME re-do these measurements, avoiding clipping
.. FIXME re-do calibration using CCFL
.. FIXME calibration for brightness imbalance due to wedge-shaped projection of spectrum
Color space mapping
*******************
Finally, to achieve the objective of measuring the LED tape's channels' precise color coordinates the measured spetra
have to be matched against the color spaces' *color matching functions*. The color matching functions describe how
strong the color space's idealized *standard observer* would react to light at a particular wavelength. Going from a
measured spectrum to color coordinates XYZ works by integrating over the product of the measurement and each color
coordinate's color matching function.
The result are three color coordinates X, Y and Z for each channel R, G and B yielding nine coordinates in total. When
written as a matrix conversion between XYZ color space and LED-RGB color space is as simple as multiplying that matrix
(or its inverse) and a vector from one of the color spaces.
In XYZ space, the set of colors that can be produced with this LED tape is described by the `parallelepiped`_ spanned by
the three channel's XYZ vectors. In the following figures, you can see a three-dimensional model of the RGB LED's color
space (colorful) as well as sRGB (white) for comparison plotted within CIE 1931 XYZ. There is no natural map to scale
both so for this illustration the LED color space has been scaled to fit. These figures were made with blender and a few
lines of python. The blender project file including all settings and the python script to generate the color space
models can be found in the `project repo`_.
.. raw:: html
<figure>
<video controls loop>
<source src="video/led_within_srgb_scale=1.0.mkv" type="video/h264">
<source src="video/led_within_srgb_scale=1.0.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space scaled to fit within XYZ with sRGB (light gray) for
comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_scale=1.0.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_scale=1.0.webm">webm download</a>
</figcaption>
</figure>
As you can see, the result is pretty disappointing. The LED's color space parallepiped is very narrow, which is because
the blue channel is much brighter than the other two channels. An easy fix for this is to scale-up the RGB space and
drop any values outside XYZ. The scaling factor is a trade-off between color space coverage and brightness. You can
produce the most colors when you clip all channels to brightness of the weakest channel (green in this case), but that
will make the result very dim. Scaling brightness like that stretches the RGB parallelepiped along its major axis. Up to
a point the number of possible colors (the gamut) increases at expense of maximum brightness. When the parallelepiped is
stretched far enought for all three channel vectors to be outside the 1,1,1 XYZ-cube, maximum brightness continues to
decrease but the gamut stays constant. I don't know a simple scientific way to solve this problem, so I just played
around with a couple of factors and settled on 2.5 as a reasonable compromise. Below is an illustration.
.. raw:: html
<figure>
<video controls loop>
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv" type="video/h264">
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space at scale factor 2.5 within XYZ with sRGB (light gray)
for comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.webm">webm download</a>
</figcaption>
</figure>
Firmware implementation
-----------------------
In the end, the above measurements yield two matrices: One for mapping XYZ to RGB, and one for mapping RGB to XYZ. Of
the several versions of CIE XYZ I chose the CIE 1931 XYZ color space as a basis for the firmware because it is most
popular. Mapping a color coordinate in one color space to the other is as simple as performing nine floating-point
multiplications and six additions. Mapping Lab or Lch to RGB is done by first mapping Lab/Lch to XYZ, then XYZ to RGB.
Lab to XYZ is somewhat complex since it requires a floating-point power for gamma correction, but any self-respecting
libc will have one of those so this is still no problem. Lch also requires floating-point sine and cosine functions, but
these should still be no problem on most hardware.
My implementation of these conversions in the ESP8266 firmware of my `Wifi LED driver`_ can be found `on Github`_. You
can view the Jupyter notebook most of the analysis above `here <http://nbviewer.jupyter.org/github/jaseg/led_drv/blob/master/doc/Spectrum%20Measurement.ipynb>`__.
.. _`on Github`: https://github.com/jaseg/esp_led_drv/blob/master/user/led_controller.c
.. _`project repo`: https://github.com/jaseg/led_drv
.. _`Wifi LED driver`: {{<ref "posts/wifi-led-driver/index.rst">}}
.. _`small driver`: {{<ref "posts/wifi-led-driver/index.rst">}}
.. _`multichannel LED driver`: {{<ref "posts/multichannel-led-driver/index.rst">}}
.. _`sRGB`: https://en.wikipedia.org/wiki/SRGB
.. _`CC BY-SA 3.0`: https://creativecommons.org/licenses/by-sa/3.0
.. _`Color spaces`: https://en.wikipedia.org/wiki/Color_space
.. _`HSV`: https://en.wikipedia.org/wiki/HSL_and_HSV
.. _`CIE Lab/LCh`: https://en.wikipedia.org/wiki/Lab_color_space
.. _`XYZ (CIE 1931)`: https://en.wikipedia.org/wiki/CIE_1931_color_space
.. _`camera obscura`: https://en.wikipedia.org/wiki/Pinhole_camera
.. _`article by two researchers from the National Science Museum in Tokyo`: http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf
.. _`spectrograph`: https://en.wikipedia.org/wiki/Ultraviolet%E2%80%93visible_spectroscopy
.. _`monochromator`: https://en.wikipedia.org/wiki/Monochromator
.. _`diffraction grating`: https://en.wikipedia.org/wiki/Diffraction_grating
.. _`SFH2701`: https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf
.. _`BPW34`: http://www.vishay.com/docs/81521/bpw34.pdf
.. _`transimpedance amplifier`: https://en.wikipedia.org/wiki/Transimpedance_amplifier
.. _`A4988`: https://www.pololu.com/file/0J450/A4988.pdf
.. _`parallelepiped`: https://en.wikipedia.org/wiki/Parallelepiped

456
content/posts/multichannel-led-driver/index.rst
View file

@ -1,456 +0,0 @@
---
title: "32-Channel LED tape driver"
date: 2018-05-02T11:31:14+02:00
---
Theoretical basics
==================
Together, a friend and I outfitted the small staircase at Berlin's Chaos Computer Club with nice, shiny RGB-WW LED tape
for ambient lighting. This tape is like regular RGB tape but with an additional warm white channel, which makes for much
more natural pastels and whites. There are several variants of RGBW tape. Cheap ones have separate RGB and white LEDs,
which is fine for indirect lighting but does not work for direct lighting. Since we wanted to mount our tape in channels
at the front of the steps, we had to use the slightly more expensive variant with integrated RGBW LEDs. These are LEDs
in the 5050 (5.0mm by 5.0mm) form factor common with RGB LEDs that have a small section divided off for the white
channel. The red, green and blue LED chips sit together in the larger section covered with clear epoxy and the white
channel is made up from the usual blue LED inside a yellow phosphor in the smaller section.
Since we wanted to light up all of 15 steps, and for greatest visual effect we would have liked to be able to control
each step individually we had to find a way to control 60 channels of LED tape with a reasonable amount of hardware.
LED tape has integrated series resistors and runs off a fixed 12V or 24V constant-voltage supply. This means you don't
need a complex constant-current driver as you'd need with high-power LEDs. You can just hook up a section of LED tape
to a beefy MOSFET to control it. Traditionally, you would do *Pulse Width Modulation* (PWM) on the MOSFET's input to
control the LED tape's brightness.
Pulse Width Modulation
----------------------
`Pulse Width Modulation`_ is a technique of controlling the brightness of a load such as an LED with a digital signal.
The basic idea is that if you turn the LED on and off much too fast for anyone to notice, you can control its power by
changing how long you turn it on versus how long you leave it off.
PWM divides each second into a large number of periods. At the beginning of each period, you turn the LED on. After
that, you wait a certain time until you turn it off. Then, you wait for the next period to begin. The periods are always
the same length but you can set when you turn off the LED. If you turn it off right away, it's off almost all the time
and it looks like it's off to your eye. If you turn it off right at the end, it's on almost all the time and it looks
super bright to your eye. Now, if you turn it off halfway into the cycle, it's on half the time and it will look to your
eye as half as bright as before. This means that you can control the LED's brightness with only a digital signal and
good timing.
.. raw:: html
<figure>
<img src="images/pwm_schema.jpg" alt="A visualization of PWM at different duty cycles.">
<figcaption>Waveforms of two PWM cycles at different duty cycles.</figcaption>
</figure>
PWM works great if you have a dedicated PWM output on your microcontroller. It's extremely simple in both hardware and
software. Unfortunately for us, controlling 32 channels with PWM is not that easy. Cheap microcontrollers only have `a
handful of hardware PWM outputs`_, so we'd either have to do everything in software, bit-banging our LED modulation, or
we'd have to use a dedicated chip.
Doing PWM in software is both error-prone and slow. Since the maximum dynamic range of a PWM signal is limited by the
shortest duty cycle it can do, software PWM being slow means it has poor PWM resolution at maybe 8 bits at most. Poor
color resolution is not a problem if all you're doing is to fade around the `HSV rainbow`_, but for ambient lighting
where you *really* want to control the brightness down to a faint shimmer you need all the color resolution you can get.
If you rule out software PWM, what remains are dedicated `hardware PWM controllers`_. Most of these have either of three
issues:
* They're expensive
* They don't have generous PWM resolution either (12 bits if you're lucky)
* They're meant to drive small LEDs such as a 7-segment display directly and you can't just hook up a MOSFET to their
output
This means we're stuck in a dilemma between two poor solutions if we'd want to do PWM. Luckily for us, PWM is not the
only modulation in town.
.. _`Pulse Width Modulation`: https://en.wikipedia.org/wiki/Pulse-width_modulation
.. _`a handful of hardware PWM outputs`: https://www.nxp.com/parametricSearch#/&c=c731_c380_c173_c161_c163&page=1
.. _`HSV rainbow`: https://en.wikipedia.org/wiki/HSL_and_HSV
.. _`hardware PWM controllers`: http://www.ti.com/lit/ds/symlink/tlc5940.pdf
Binary Code Modulation
----------------------
PWM is the bread-and-butter of the maker crowd. Everyone and their cat is doing it and it works really well most of the
time. Unbeknownst to most of the maker crowd, there is however another popular modulation method that's mostly used in
professional LED systems: Enter `*Binary Code Modulation* (BCM) <http://www.batsocks.co.uk/readme/art_bcm_1.htm>`_.
BCM is to PWM sort of what barcodes are to handwriting. While PWM is easy to understand and simple to implement if all
you have is a counter and an IO pin, BCM is more complicated. On the other hand, computers can do complicated and BCM
really shines in multi-channel applications.
Similar to PWM, BCM works by turning on and off the LED in short periods fast enough to make your eye perceive it as
partially on all the time. In PWM the channel's brightness is linearly dependent on its duty cycle, i.e. the percentage
it is turned on. In PWM the duty cycle D is the total period T divided by the on period T_on. The issue with doing PWM
on many channels at once is that you have to turn off each channel at the exact time to match its duty cycle.
Controlling many IO pins at once with precise timing is really hard to do in software.
BCM avoids this by further dividing each period into smaller periods which we'll call *bit periods* and splitting each
channel's duty cycle into chunks the size of these bit periods. The amazingly elegant thing in BCM now is that as you
can guess from the name these bit periods are weighted in powers of two. Say the shortest bit period lasts 1
microsecond. Then the second-shortest bit period is 2 microseconds and the third is 4, the fifth 8, the sixth 16 and so
on.
.. raw:: html
<figure>
<img src="images/bcm_schema.jpg" alt="A visualization of BCM at different duty cycles.">
<figcaption>Waveforms of a single 4-bit BCM cycle at different duty cycles. This BCM can produce 16 different
levels.</figcaption>
</figure>
Staggered like this, you turn on the LED for integer value of microseconds by turning it on in the bit periods
corresponding to the binary bits of that value. If I want my LED to light for 19 microseconds every period, I turn it on
in the 16 microsecond bit period, the 2 microsecond bit period and the 1 microsecond bit period and leave it off for the
4 and 8 mircosecond bit periods.
Now, how this is better instead of just more complicated than plain old PWM might not be clear yet. But consider this:
Turning on and off a large number of channels, each at its own arbitrary time is hard because doing the timing in
software is hard. We can't use hardware timers since we only have two or three of those, and we have 32 channels.
However, we can use one hardware timer to trigger a really cheap external latch to turn on or off the 32 channels all at
once. With this setup, we can only controll all channels at once, but we can do so with very precise timing.
All we need to do is to set our timer to the durations of the BCM bit periods, and we can get the same result as we'd
get with PWM with only one hardware timer and a bit of code that is not timing-critical anymore.
Applications of Binary Code Modulation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BCM is a truly wondrous technique, and outside of hobbyist circles it is in fact very widely known. Though we're using
it to control just 32 channels here, you can do much more channels without any problems. The most common application
where BCM is invariably used is *any* kind of LED screen. Controlling the thousands and thousands of LEDs in an LED
screen with PWM with a dedicated timer for each LED would not be feasible. With BCM, all you need to dedicate to a
single LED is a flipflop (or part of one if you're multiplexing). In fact, there is a whole range of `ICs with no other
purpose than to enable BCM on large LED matrices <http://www.vabolis.lt/stuff/MBI5026.pdf>`_. Basically, these are a
high-speed shift register with latched outputs much like the venerable 74HC595_, only their outputs are constant-current
sinks made so that you can directly connect an LED to them.
.. _74HC595: http://www.ti.com/lit/ds/symlink/sn74hc595.pdf
Running BCM on LED tape
~~~~~~~~~~~~~~~~~~~~~~~
In our case, we don't need any special driver chips to control our LED tape. We just connect the outputs of a 74HC595_
shift register to one MOSFET_ each, and then we directly connect the LED tape to these MOSFETs. The MOSFETs allow us to
drive a couple of amps into the LED tape from the weak outputs of the shift register.
The BCM timing is done by hooking up two timer channels of our microcontroller to the shift registers *strobe* and
*reset* inputs. We set the timer to PWM mode so we can generate pulses with precise timing. At the beginning of each
bit period, a pulse will strobe the data for this bit period that we shifted in previously. At the end of the bit
period, one pulse will reset the shift register and one will strobe the freshly-reset zeros into the outputs.
.. raw:: html
<figure>
<img src="images/olsndot_output_schematic.jpg" alt="From left to right, we see the STM32, one of the shift
registers, and the LEDs and MOSFETs. The LED tape is driven to ground by the MOSFETs, which are in turn directly
driven from the shift register outputs. The shift register is wired up to the STM32 with its clock and data
inputs on SCK and MOSI and its RESET and STROBE inputs on channel 2 and 3 of timer 1.">
<figcaption>
The schematic of a single output of this LED driver. Multiple shift register stages can be cascaded.
</figcaption>
</figure>
Our implementation of this system runs on an STM32F030F4P6_, the smallest, cheapest ARM microcontroller you can get from
ST. This microcontroller has only 16kB of flash and 1kB of RAM, but that's plenty for our use. We use its SPI controller
to feed the modulation data to the shift registers really fast, and we use two timer channels to control the shift
registers' reset and strobe.
We can easily cascade shift registers without any ill side-effects, and even hundreds of channels should be no problem
for this setup. The only reason we chose to stick to a 32-channel board is the mechanics of it. We thought it would be
easier to have several small boards instead of having one huge board with loads of connectors and cables coming off it.
The BOM cost per channel for our system is 3ct for a reasonable MOSFET, about 1ct for one eighth of a shift register
plus less than a cent for one resistor between shift register and MOSFET. In the end, the connectors are more expensive
than the driving circuitry.
.. _MOSFET: https://en.wikipedia.org/wiki/MOSFET
.. _STM32F030F4P6: http://www.st.com/resource/en/datasheet/stm32f030f4.pdf
Hardware design
===============
From this starting point, we made a very prototype-y hardware design for a 32-channel 12V LED tape driver. The design is
based on the STM32F030F4P6_ driving the shift registers as explained above. The system is controlled through an RS485_
bus that is connected up to the microcontroller's UART using an MAX485_-compatible RS485 transceiver. The LED tape is
connected using 9-pin SUB-D_ connectors since they are cheap and good enough for the small current of our short segments
of LED tape. The MOSFETs we use are small SOT-23_ logic-level MOSFETs. In various prototypes we used both International
Rectifier's IRLML6244_ as well as Alpha & Omega Semiconductor's AO3400_. Both are good up to about 30V/5A. Since we're
only driving about 2m of LED tape per channel we're not going above about 0.5A and the MOSFETs don't even get warm.
.. _RS485: https://en.wikipedia.org/wiki/RS-485
.. _MAX485: https://datasheets.maximintegrated.com/en/ds/MAX1487-MAX491.pdf
.. _IRLML6244: https://www.infineon.com/dgdl/?fileId=5546d462533600a4015356686fed261f
.. _AO3400: http://aosmd.com/pdfs/datasheet/AO3400.pdf
.. _SUB-D: https://en.wikipedia.org/wiki/D-subminiature
.. _SOT-23: http://www.nxp.com/documents/outline_drawing/SOT23.pdf
Switching nonlinearities
------------------------
During testing of our initial prototype, we noticed that the brightness seemed to jump around when fading to very low
values. It turned out that our extremely simple LED driving circuit consisting of only the shift register directly
driving a MOSFET, which in turn directly drives the LED tape was maybe a little bit too simple. After some measurements
it turned out that we were looking at about 6Vpp of ringing on the driver's output voltage. The picture below is the
voltrage we saw on our oscilloscope on the LED tape.
.. raw:: html
<figure>
<img src="images/driver_ringing_strong.jpg" alt="Strong ringing on the LED voltage waveform edge at about
100% overshoot during about 70% of the cycle time.">
<figcaption>Bad ringing on the LED output voltage caused by wiring inductance. Note that the effect on the
actual LED current is less bad than this looks since the LED's V/I curve is nonlinear.</figcaption>
</figure>
Dynamic switching behavior: Cause and Effect
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A bit of LTSpice_ action later we found that the inductance of the few metres of cable leading to the LED tape is the
likely culprit. The figure below is the schematic used for the simulations.
.. raw:: html
<figure>
<img src="images/driver_output_ltspice_schematic.jpg" alt="The LTSpice schematic of one output of the driver,
taking into account the shift register's output ESR and the wiring ESL.">
<figcaption>The schematic of the simulation in LTSpice</figcaption>
</figure>
As tested, the driver does not include any per-output smoothing so the ~.5A transient on each BCM cycle hits the cable
in full. Combined with the cable inductance, this works out to a considerable lag of the rising edge of the LED
current, and bad ringing on its falling edge. Below is the voltage on the LED output from an LTSpice simulation of our
driver.
.. raw:: html
<figure>
<img src="images/overshoot_sim_r0.svg" alt="The result of the LTSpice simulation of our driver output. The LED
current shows similar ringing to what we measured using the oscilloscope. Interestingly, the gate voltage shows
strong ringing, too.">
<figcaption>The result of our LTSpice simulation. This simulation assumes 1µH of wiring inductance and 50Ω of
output impedance on the part of the shift register. The ringing at the gate visible in the gate voltage graph is
due to feed-through of the ringing at the output through the MOSFET's parasitic Cgd.</figcaption>
</figure>
We were able to reduce the rining and limit the effect somewhat by putting a 220Ω series resistor in between the shift
register output and the MOSFET gate. This resistor forms an RC circuit with the MOSFET's nanofarad or two of gate
capacitance. The result of this is that the LED current passing the wire's ESL rises slightly more slowly and thus the
series inductance gets excited slightly less, and the overshoot decreases. Below is a picture of the waveform with the
damping resistor in place and a picture of our measurement for comparison. The resistor values don't agree perfectly
since the estimated ESL and stray capacitance of the wiring is probably way off.
.. raw:: html
<figure>
<img src="images/driver_ringing_weak.jpg" alt="Weak ringing on the LED voltage waveform edge at about 30%
overshoot during about 20% of the cycle time.">
<figcaption>Adding a resistor in front of the MOSFET gate to slow the transition damped the ringing somewhat,
but ultimately it cannot be eliminated entirely. Note how you can actually see the miller plateau on the
trailing edge of this signal.
</figcaption>
</figure>
.. raw:: html
<figure>
<img src="images/overshoot_sim_r100.svg" alt="The result of the LTSpice simulation of our driver output with an
extra 100 Ohms between shift register output and MOSFET gate. Similar to the oscilloscope measurement the
ringing is much reduced in its amplitude.">
<figcaption>The LTSpice simulation result with the same parameters as above but with an extra 100Ω between the
shfit register's output and the MOSFET's gate.</figcaption>
</figure>
A side effect of this fix is that now the effective on-time of the LED tape is much longer than the duty cycle at the
shift register's output at very small duty cycles (1µs or less). This is caused by the MOSFET's `miller
plateau`_. For illustration, below is a graph of both the excitation waveform (the boxy line) and the resulting LED
current (the other ones) both without damping (top) and with 220Ω damping (bottom). As you can see the effective duty
cycle of the LED current is not at all equal to the 50% duty cycle of the excitation square wave.
.. raw:: html
<figure>
<img src="images/asymmetric_iled.svg" alt="The result of an LTSpice simulation of the LED duty cycle without and
with damping. Dampening widens the LED current waveform from 50% duty cycle with sharp edges to about 80% duty
cycle with soft edges.">
<figcaption>Simulated LED duty cycle with and without damping. The damping resistance used in this simulation
was 220Ω.</figcaption>
</figure>
.. raw:: html
<figure>
<img src="images/asymmetric_vgate.svg" alt="The gate voltages in the spice simulation above. The undamped
response shows sharp edges with the miller plateau being a barely noticeable step, but with strong ringing on
the trailing edge. The damped response shows RC-like slow-edges, but has wide miller plateaus on both edges
adding up to about 50% of the pulse width.">
<figcaption>The MOSFET gate voltage from the simulation in the figure above. You can clearly see how the miller
plateau (the horizontal part of the trace at about 1V) is getting much wider with added damped, and how the
resulting gate charge/discharge curve is not at all that of a capacitor anymore.</figcaption>
</figure>
In conclusion, we have three major causes for our calculated LED brightness not matching reality:
* Ringing of the equivalent series inductance of the wiring leading up to the LED tape
* Miller plateau lag
* The damping resistor and the MOSFET gate forming an RC filter that helps with wire ESL ringing but worsens the miller
plateau issue and deforms the LED current edges.
Added up, these three effects yield a picture that agrees well with our simulations and measurements. The overall effect
is neglegible at long period durations (>10µs), but gets really bad at short period durations (<1µs). The effect is
non-linear, so correcting for it is not as simple as adding an offset.
.. _LTSpice: http://www.analog.com/en/design-center/design-tools-and-calculators/ltspice-simulator.html
.. _`miller plateau`: https://www.vishay.com/docs/68214/turnonprocess.pdf
Measuring LED tape brightness
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to correct for the nonlinearities mentioned above, we decided to implement a lookup table mapping BCM period to
actual timer setting. That is, each row of the table contains the actual period length we need to set the
microcontroller's timer to in order to get our intended brightness steps.
To calibrate our driver, we needed a setup for reproducible measurement of the relative brightness of our LED tape at
different settings. Absolute brightness is not of interest to us as the eye can't perceive it. To perform the
calibration, the LED driver is set to enable each single BCM period in turn, i.e. brightness values 1, 2, 4, 8, 16 etc.
The setup we used to measure the LED tape's brightness consists of a bunch of LED tape stuck into a tin can for
shielding against both stray light and electromagnetic interference and a photodiode looking at the LED tape. We used
the venerable BPW34_ photodiode in our setup as I had a bunch leftover from another project and because they are quite
sensitive owing to their physically large die area.
.. raw:: html
<figure>
<img src="images/linearization_setup.jpg" alt="The led measurement setup consists of several PCBs and a
breadboard linked with a bunch of wires and a big tin can to shield the LEDs and the photodiode. A large sub-D
connector is put into the top of the tin can as a feed-through for the LED tape's control signals and the
photodiode signal. In the background the control laptop is visible.">
<figcaption>The LED brighness measurement setup. The big tin can contains a bunch of LED tape and the
photodiode. The breadboard on the right is used for the photodiode preamplifier and for jumpering around the LED
tape's channels. The red board next to it is the buspirate used as ADC. The board on the bottom left is a
TTL-to-RS485 converter and the board in the middle is the unit under test.</figcaption>
</figure>
The photodiode's photocurrent is converted into a voltage using a very simple transimpedance amplifier based around a
MCP6002_ opamp that was damped into oblivion with a couple nanofarads of capacitance in its feedback loop. The MCP6002_
is a fine choice here since I had a bunch and because it is a CMOS opamp, meaning it has low bias current that would
mess up our measurements. For many applications, opamp bias current is not a big issue but when using the opamp to
directly measure very small currents at its input it quickly swamps out the signal for most BJT-input types.
The transimpedance amplifier's output is read from the computer using the ADC input of a buspirate USB thinggamajob. In
general I would not recommend the buspirate as a tool for this job since it's ADC is not particularly good and it's
programming interface is positively atrocious, but it was what I had and it beat first wiring up one of the dedicated
ADC chips I had in my parts bin.
The computer runs a small python script cycling the LED tape through all its BCM period settings and taking a brightness
measurement at each step. Later on, these measurements can be plotted to visualize the resulting slope's linearity, and
we can even do a simulation of the resulting brightness for all possible control values by just adding the measured
photocurrents for a certain BCM setpoint just as our retinas would do.
.. raw:: html
<figure>
<img src="images/driver_linearity_raw.svg" alt="">
<figcaption>
A plot of the measured brightness of our LED tape for each BCM period. The brightness values are normalized
to the value measured at the LSB setpoint (brightness=1/65535). Ideally, this plot would show a straight
line with slope 1. Obviously, it doesn't. The bend in the curve is caused by the above-mentioned duty cycle
offset adding an offset to all brightness values. Shown is both the raw data (light), which has essentially zero
measurement error and a linear fit (dark).
The plot is in log-log to approximate how the human eye would perceive brightness, i.e. highly sensitive at
low values but not very sensitive at all at large values.
</figcaption>
</figure>
While it would be possible to fully automate the optimization of BCM driver lookup tables, we needed only one and in the
end I just sat down and manually tweaked the ideal values we initially calculated until I liked the result. You can see
the resulting brightness curve below.
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/uncorrected_brightness_sim.svg" alt="">
<figcaption>
Calculated brightness curve for the uncorrected BCM setup. As you can see, at low setpoints the result
is about as smooth as sandpaper, which is well in line with our observations. At high setpoints the
offset gets swamped out and the nonlinearity in the low bits is not visible anymore.
</figcaption>
</figure><figure class="side-by-side">
<img src="images/corrected_brightness_sim.svg" alt="">
<figcaption>
Brightness curve for the corrected BCM setup extrapolated using actual measurements. Looks as buttery
smooth in real life as it does in this plot.
</figcaption>
</figcaption>
</figure>
</figure>
.. _BPW34: http://www.vishay.com/docs/81521/bpw34.pdf
.. _MCP6002: http://ww1.microchip.com/downloads/en/DeviceDoc/21733j.pdf
Controlling the driver
----------------------
Now that our driver was behaving linear enough that you couldn't see it actually wasn't we needed a nice way to control
it from a computer of our choice. In the ultimate application (our staircase) we'll use a raspberry pi for this. Since
we already settled on an RS485_ bus for its robustness and simplicity, we had to device a protocol to control the driver
over this bus. Here, we settled on a simple, COBS_-based protocol for the reasons I wrote about in `How to talk to your
microcontroller over serial <serial-protocols>`_.
To address our driver nodes, we modified the Makefile to build a random 32-bit MAC into each firmware image. The
protocol has only five message types:
1. A 0-byte *ping* packet, to which each node would reply with its own address in the
first 100ms after boot. This can be used to initially discover the addresses of all nodes connected to the bus. You'd
spam the bus with *ping* packets, and then hit reset on each node in turn. The control computer would then receive
each device's MAC address as you hit reset.
2. A 4-byte *address* packet that says which device that the following packet is for. This way of us using the packet
length instead of a packet type field is not particularly elegant, but our system is simple enough and it was easy to
implement.
3. A 64-byte *frame buffer* packet that contains 16 bits of left-aligned brightness data for every channel
4. A one-byte *get status* packet that tells the device to respond with...
5. ...a 27-byte status packet containing a brief description of the firmware (version number, channel count, bit depth
etc.) as well as the device's current life stats (VCC, temperature, uptime, UART frame errors etc.).
Wrapped up in a nice python interface we can now easily enumerate any drivers we connect to a bus, query their status
and control their outputs.
.. _COBS: https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing
Conclusion
----------
.. raw:: html
<figure>
<figure class="side-by-side">
<a href="images/olsndot_schematic.png">
<img src="images/olsndot_schematic.png" alt="A picture of the LED driver schematic">
</a>
<figcaption>The LED driver <a href="images/olsndot_schematic.png">schematic</a></figcaption>
</figure><figure class="side-by-side">
<a href="images/olsndot_pcb.png">
<img src="images/olsndot_pcb.png" alt="A picture of the LED driver PCB layout">
</a>
<figcaption>The LED driver <a href="images/olsndot_pcb.png">PCB layout</a></figcaption>
</figure>
</figure>
Putting some thought into the control circuitry and software, you can easily control large numbers of channels of LEDs
using extremely inexpensive driving hardware without any compromises on dynamic range. The design we settled on can
drive 32 channels of LED tape with a dynamic range of 14bit at a BOM cost of below 10€. All it really takes is a couple
of shift registers and a mildly bored STM32 microcontroller.
Get a PDF file of the schematic and PCB layout `here <olsndot_v02_schematics_and_pcb.pdf>`__ or download the CAD files
and the firmware sources `from github <https://github.com/jaseg/led_drv>`_. You can view the Jupyter notebook used to
analyze the brightness measurement data `here <http://nbviewer.jupyter.org/github/jaseg/led_drv/blob/master/doc/Run_analysis.ipynb>`__.

38
content/posts/private-contact-discovery/index.rst
View file

@ -1,38 +0,0 @@
---
title: "Private Contact Discovery"
date: 2019-06-22T10:30:00+08:00
---
Private Contact Discovery
=========================
Private Contact Discovery (PCD) is the formal name for the problem modern smartphone messenger applications have on
installation: Given a user's address book, find out which of their contacts also use the same messenger without the
messenger's servers learning anything about the user's address book. The widespread non-private way to do this is to
simply upload the user's address book to the app's operator's servers and do an SQL JOIN keyed on the phone number field
against the database of registered users. People have tried sprinkling some hashes over these phone numbers in an
attempt to improve privacy, but obviously running a brute-force preimage attack given a domain of maybe a few billion
valid inputs is not cryptographically hard.
Private Contact Discovery can be phrased in terms of Private Set Intersection (PSI), the cryptographic problem of having
two parties holding one set each find the intersection of their sets without disclosing any other information. PSI has
been an active field of research for a while and already yielded useful results for some use cases. Alas, none of those
results were truly practical yet for usage in PCD in a typical messenger application. They would require too much CPU
time or too much data to be transferred.
At USENIX Security 2019, Researchers from technical universities Graz and Darmstadt published a paper titled *Private
Contact Discovery at Scale*
(`eprint <https://eprint.iacr.org/2019/517>`__ | `PDF <https://eprint.iacr.org/2019/517.pdf>`__).
In this paper, they basically optimize the hell out of existing cryptographic solutions to private contact discovery,
jumping from a still-impractical state of the art right to practicality. Their scheme allows a client with 1k contacts
to run PCD against a server with 1B contacts in about 3s on a phone. The main disadvantage of their scheme is that it
requires the client to in advance download a compressed database of all users, that clocks in at about 1GB for 1B users.
I found this paper very interesting for its immediate practical applicability. As an excuse to dig into the topic some
more, I gave a short presentation at my university lab's research seminar on this paper
(slides: `PDF <mori_semi_psi_talk.pdf>`__ | `ODP <mori_semi_psi_talk.odp>`__).
Even if you're not working on secure communication systems on a day-to-day basis this paper might interest you. If
you're working with social account information of any kind I can highly recommend giving it a look. Not only might your
users benefit from improved privacy, but your company might be able to avoid a bunch of data protection and
accountability issues by simply not producing as much sensitive data in the first place.

249
content/posts/serial-protocols/index.rst
View file

@ -1,249 +0,0 @@
---
title: "How to talk to your microcontroller over serial"
date: 2018-05-19T08:09:46+02:00
---
Scroll to the end for the `TL;DR <Conclusion_>`_.
In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will
summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error
conditions.
If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to
another microcontroller or to a computer. In the age of USB, a serial port is still the simplest and quickest way to get
communication to a control computer up and running. Integrating a ten thousand-line USB stack into your firmware and
writing the necessary low-level drivers on the host side might take days. Poking a few registers to set up your UART to
talk to an external hardware USB to serial converter is a matter of minutes.
This simplicity is treacherous, though. Oftentimes, you start writing your serial protocol as needs arise. Things might
start harmless with something like ``SET_LED ON\n``, but unless you proceed it is easy to end up in a hot mess of command
modes, protocol states that breaks under stress. The ways in which serial protocols break are manifold. The simplest one
is that at some point a character is mangled, leading to both ends of the conversation ending up in misaligned protocol
states. With a fragile protocol, you might end up in a state that is hard to recover from. In extreme cases, this leads
to code such as `this gem`_ performing some sort of arcane ritual to get back to some known state, and all just because
someone did not do their homework. Below we'll embark on a journey through the lands of protocol design, exploring the
facets of this deceptively simple problem.
.. _`this gem`: https://github.com/juhasch/pyBusPirateLite/blob/master/pyBusPirateLite/BBIO_base.py#L68
Text-based serial protocols
===========================
The first serial protocol you've likely written is a human-readable, text-based one. Text-based protocols have the big
advantage that you can just print them on a terminal and you can immediately see what's happening. In most cases you can
even type out the protocol with your bare hands, meaning that you don't really need a debugging tool beyond a serial
console.
However, text-based protocols also have a number of disadvantages. Depending on your application, these might not matter
and in many cases a text-based protocol is the most sensible solution. But then, in some cases they might and it's good
to know when you hit one of them.
Problems
--------
Low information density
~~~~~~~~~~~~~~~~~~~~~~~
Generally, you won't be able to stuff much more than four or five bit of information down a serial port using a
human-readable protocol. In many cases you will get much less. If you have 10 commands that are only issued a couple
times a second nobody cares that you spend maybe ten bytes per command on nice, verbose strings such as ``SET LED``. But
if you're trying to squeeze a half-kilobyte framebuffer down the line you might start to notice the difference between
hex and base-64 encoding, and a binary protocol might really be more up to the job.
Complex parsing code
~~~~~~~~~~~~~~~~~~~~
On the computer side of thing, with the whole phalanx of an operating system, the standard library of your programming
language of choice and for all intents and purposes unlimted CPU and memory resources to spare you can easily parse
anything spoken on a serial port in real time, even at a blazing fast full Megabaud. The microcontroller side however is
an entirely different beast. On a small microcontroller, printf_ alone will eat about half your flash. On most small
microcontrollers, you just won't get a regex library even though it would make parsing textual commands *so much
simpler*. Lacking these resources, you might end up hand-knitting a lot of low-level C code to do something seemingly
simple such as parsing ``set_channel (13, 1.1333)\n``. These issues have to be taken into account in the protocol design
from the beginning. If you don't really need matching parentheses, don't use them.
Fragile protocol state
~~~~~~~~~~~~~~~~~~~~~~
Say you have a ``SET_DISPLAY`` command. Now say your display can display four lines of text. The obvious approach to this
is probably the SMTP_ or HTTP_ way of sending ``SET_DISPLAY\nThis is line 1\nThis is line 2\n\n``. This would certainly
work, but it is very fragile. With this protocol, you're in trouble if at any point the terminating second newline
character gets mangled (say, someone unplugs the cable, or the control computer reboots, or a cosmic ray hits something
and ``0x10 '\n'`` turns into ``0x50 'P'``).
.. _SMTP: https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol
.. _HTTP: https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol
Timeouts don't work
~~~~~~~~~~~~~~~~~~~
You might try to solve the problem of your protocol state machine tangling up with a timeout. "If I don't get a valid
command for more than 200ms I go back to default state." But consider the above example. Say, your control computer
sends a ``SET_DISPLAY`` command every 100ms. If in one of them the state machine tangles up, the parser hangs since the
timeout is never hit, a new line of text arriving every 100ms.
Framing is hard
~~~~~~~~~~~~~~~
You might also try to drop the second newline and using a convention such as ``SET_DISPLAY`` is followed by two lines of
text, then commands resume.". This works as long as your display contents never look like commands. If you are only ever
displaying the same three messages on a character LCD that might work, but if you're displaying binary framebuffer
data you've lost.
Solutions
---------
Keep the state machine simple
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Always use a single line of text to represent a single command. Don't do protocol states or modes where you can toggle
between different interpretations for a line. If you have to send human-readable text as part of a command (such as
``SET_DISPLAY``) escape it so it doesn't contain any newlines.
This way, you keep your protocol state machine simple. If at any time your serial trips and flips a bit or looses a byte
your protocol will recover on the next newline character, returning to its base state.
Encode numbers in hex when possible
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Printing a number in hexadecimal is a very tidy operation, even on the smalest 8-bit microcontrollers. In contrast,
printing decimal requires both division and remainder in a loop which might get annoyingly code- and time-intensive on
large numbers (say a 32-bit int) and small microcontrollers.
If you have to send fractional values, consider their precision. Instead of sending a 12 bit ADC result as a 32-bit
float formatted like ``0.176513671875`` sending ``0x2d3`` and dividing by 4096 on the host might be more sensible. If you
really have to communicate big floats and you can't take the overhead of including both printf_ and scanf_ you can
use hexadecimal floating point, which is basically ``hex((int)foo) + "." + hex((int)(65536*(foo - (int)foo)))`` for four
digits. You can also just hex-encode the binary IEEE-754_ representation of the float, sending ``hex(*(int *)&float)``.
Most programming languages will have a `simple, built-in means to parse this sort of thing
<https://docs.python.org/3.5/library/struct.html>`__.
.. _printf: http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c
.. _scanf: http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfscanf.c
.. _IEEE-754: https://en.wikipedia.org/wiki/IEEE_754
Escape multiline strings
~~~~~~~~~~~~~~~~~~~~~~~~
If you have to send arbitrary strings, escape special characters. This not only has the advantage of yielding a robust
protocol: It also ensures you can actually see everything that's going on when debugging. The string ``"\r\n"`` is easy to
distinguish from ``"\n"`` while your terminal emulator might not care.
The simplest encoding to use is the C-style backslash encoding. Host-side, most languages will have a `built-in means of
escaping a string like that <https://docs.python.org/3.5/library/codecs.html#text-encodings>`__.
Encoding binary data
--------------------
For binary data, hex and base-64 are the most common encodings. Since hex is simpler to implement I'd go with it unless
I really need the 30% bandwidth improvement base-64 brings.
Binary serial protocols
=======================
In contrast to anything human-readable, binary protocols are generally more bandwidth-efficient and are easier to format
and parse. However, binary protocols come with their own version of the caveats we discussed for text-based protocols.
The framing problem in binary protocols
---------------------------------------
The most basic problems with binary protocols as with text-based ones is framing, i.e. splitting up the continuous
serial data stream into discrete packets. The issue is that it is that you have to somehow mark boundaries between
frames. The simplest way would be to use some special character to delimit frames, but then any 8-bit character you
could choose could also occur within a frame.
SLIP/PPP-like special character framing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Some protocols solve this problem much like we have solved it above for strings in line-based protocols, by escaping any
occurence of the special delimiter character within frames. That is, if you want to use ``0x00`` as a delimiter, you would
encode a packet containing ``0xde 0xad 0x00 0xbe 0xef`` as something like ``0xde 0xad 0x01 0x02 0xbe 0xef``, replacing the
null byte with a magic sequence. This framing works, but is has one critical disadvantage: The length of the resulting
escaped data is dependent on the raw data, and in the worst case twice as long. In a raw packet consisting entirely of
null bytes, every byte must be escaped with two escape bytes. This means that in this case the packet length doubles,
and in this particular case we're even less efficient than base-64.
Highly variable packet length is also bad since it makes it very hard to make any timing guarantees for our protocol.
9-bit framing
~~~~~~~~~~~~~
A framing mode sometimes used is to configure the UARTs to transmit 9-bit characters and to use the 9th bit to designate
control characters. This works really well, and gives plenty of control characters to work with. The main problem with
this is that a 9-bit serial interface is highly nonstandard and you need UARTs on both ends that actually support this
mode. Another issue is that though more efficient than both delmitier-based and purely text-based protocols, it still
incurs an extra about 10% of bandwidth overhead. This is not a lot if all you're sending is a little command every now
and then, but if you're trying to push large amounts of data through your serial it's still bad.
COBS
~~~~
Given the limitations of the two above-mentioned framing formats, we really want something better. The `Serial Line
Internet Protocol (SLIP)`_ as well as the `Point to Point Protocol (PPP)`_, standardized in 1988 and 1994 respectively,
both use escape sequences. This might come as a surprise, but humanity has actually still made significant technological
progress on protocols for 8-bit serial interfaces until the turn of the millennium. In 1999, `Consistent Overhead Byte
Stuffing (COBS)`_ (`wiki <https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing>`__) was published by a few
researchers from Apple Computer and Stanford University. As a reaction on the bandwidth doubling problem present in
PPP_, COBS *always* has an overhead of a single byte, no matter what or how long a packet's content is.
COBS uses the null byte as a delimiter interleaves all the raw packet data and a `run-length encoding`_ of the non-zero
portions of the raw packet. That is, it prepends the number of bytes until the first zero byte to the packet, plus one.
Then it takes all the leading non-zero bytes of the packet, unmodified. Then, it again encodes the distance from the
first zero to the second zero, plus one. And then it takes the second non-zero run of bytes unmodified. And so on. At
the end, the packet is terminated with a zero byte.
The result of this scheme is that the encoded packet does not contain any zero bytes, as every zero byte has been
replaced with the number of bytes until the next zero byte, plus one, and that can't be zero. Both formatter and parser
each have to keep a counter running to keep track of the distances between zero bytes. The first byte of the packet
initializes that counter and is dropped by the parser. After that, every encoded byte received results in one raw byte
parsed.
While this might sound more complicated than the escaping explained above, the gains in predictability and efficiency
are worth it. An implementation of encoder and decoder should each be about ten lines of C or two lines of Python. A
minor asymmetry of the protocol is that while decoding can be done in-place, encoding either needs two passes or you
need to scan forward for the next null byte.
.. _`Point to Point Protocol (PPP)`: https://en.wikipedia.org/wiki/Point-to-Point_Protocol
.. _PPP: https://en.wikipedia.org/wiki/Point-to-Point_Protocol
.. _`Serial Line Internet Protocol (SLIP)`: https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol
.. _`Consistent Overhead Byte Stuffing (COBS)`: http://www.stuartcheshire.org/papers/COBSforToN.pdf
.. _`Point-to-Point Protocol (PPP)`: https://en.wikipedia.org/wiki/Point-to-Point_Protocol
.. _`run-length encoding`: https://en.wikipedia.org/wiki/Run-length_encoding
State machines and error recovery
---------------------------------
In binary protocols even more than in textual ones it is tempting to build complex state machines triggering actions on
a sequence of protocol packets. Please resist that temptation. As with textual protocols keeping the protocol state to
the minimum possible allows for a self-synchronizing protocol. A serial protocol should be designed such that if due to
a dropped packet or two both ends will naturally re-synchronize within another packet or two. A simple way of doing that
is to always transmit one semantic command per packet and to design these commands in the most idempotent_ way possible.
For example, when filling a framebuffer piece by piece, include the offset in each piece instead of keeping track of it
on the receiving side.
.. _idempotent: https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning
Conclusion
==========
Here's your five-step guide to serial bliss:
1. Unless you have super-special requirements, always use the slowest you can get away with from 9600Bd, 115200Bd or
1MBd. 8N1 framing if you're talking to anything but another microcontroller on the same board. These settings are
the most common and cover any use case. You'll inevitably have to guess these at some point in the future.
2. If you're doing something simple and speed is not a particular concern, use a human-readable text-based protocol. Use
one command/reply per line, begin each line with some sort of command word and format numbers in hexadecimal. You get
bonus points if the device replies to unknown commands with a human-readable status message and prints a brief
protocol overview on boot.
3. If you're doing something even slightly nontrivial or need moderate throughput (>1k commands per second or >20 byte of
data per command) use a COBS-based protocol. If you don't have a better idea, go for an ``[target MAC][command
ID][command arguments]`` packet format for multidrop busses. For single-drop you may decide to drop the MAC address.
4. Always include some sort of "status" command that prints life stats such as VCC, temperature, serial framing errors
and uptime. You'll need some sort of ping command anyway and that one might as well do something useful.
5. If at all possible, keep your protocol context-free across packets/lines. That is, a certain command should always be
self-contained, and no command should change the meaning of the next packet or line that is sent. This is really
important to allow for self-synchronization. If you really need to break up something into multiple commands, say you
want to set a large framebuffer in pieces, do it in a idempotent_ way: Instead of sending something like ``FRAMEBUFFER
INCOMING:\n[byte 0-16]\n[byte 17-32]\n[...]\nEND OF FRAME`` rather send ``FRAMEBUFFER DATA FOR OFFSET 0: [byte
0-16]\nFRAMEBUFFER DATA FOR OFFSET 17: [byte 17-32]\n[...]\nSWAP BUFFERS\n``.

BIN
content/posts/sybil-resistance-identity/images/succulents.jpg
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 574 KiB

244
content/posts/sybil-resistance-identity/index-old.rst
View file

@ -1,244 +0,0 @@
---
title: "Theia Attack Resistance and Digital Identity"
date: 2020-09-09T15:00:00+02:00
---
.. raw:: html
<figure class="header">
<img src="images/succulents.jpg">
<figcaption>Photo by <a href="https://unsplash.com/@timbennettcreative">Tim Bennett</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption>
</figure>
Theia in Cyberspace
===================
In informatics, the term *distributed system* is used to describe the aggregate behavior of a complex network made up of
individual computers. For decades, computer scientists to some success have been trying to figure out how exactly the
individual computers that make up such a distributed system need to be programmed for the resulting amalgamation to
behave in a predictable, maybe even a desirable way. Though seemingly simple on its surface, this problem has a
surprising depth to it that has yielded research questions for a whole field for several decades now. One particular
as-of-yet unsolved problem is resistance against *theia attacks* (or "sybil" attacks in older terminology).
Named after the 1973 book by Flora Rheta Schreiber on dissociative identity disorder, a sybil attack is an
attack where one computer in a distributed system pretends to be multiple computers to gain an advantage. From your
author's standpoint, naming a type of computer security attack after a medical condition was an unfortunate choice.
For this reason this post uses the term *Theia attack* to refer to the same concept. Theia is a greek godess of light
and glitter and the name alludes to the attacker performing something alike an optical illusion, causing the attacked
to perceive multiple distinct images that in the end are all only reflections of the same attacker.
The core insight of computer science research on theia attacks is that there cannot be any technological way of
preventing such an attack, and any practical countermeasure must be grounded in some authority or ground truth that is
external to the systems—bridging from technology to its social or political context.
Looking around, we can see a parallel between this question ("which computer is a real computer?") and a social issue
that recently has been growing in importance: Just like computers can pretend to be other computers, they can also
pretend to be humans. As can humans. Be it within the context of election manipulation or down-to-earth astroturfing_
the recurring issue is that in today's online communities, it is hard for an individual to tell who of their online
acquaintances are who they seem to be. Different platforms attempt different solutions to this problem, and all fail in
some way or another. Facebook employs good old snitching, turning people against each other and asking them "Do you know
this person?". Twitter is more laid-back and avoids this Stasi_ methodology in favor of requiring a working mobile phone
number from its subjects, essentially short-circuiting identity verification to the phone company's check of their
subscriber's national passport.
.. the preceding is a simplified representation of these platform's practices. In particular facebook uses several
methods depending on the case. I think this abbreviated discussion should be ok for the sake of the argument. I am
not 100% certain on the accuracy on the accuracy of the statement though. Does fb still do the snitching thing? Is
twitter usually content with a phone number?
Trusting Crypto-Anarchist Authorities
=====================================
Beyond these centralistic solutions to the problem, crypto-anarchists and anarcho-capitalists have been brewing on some
interesting novel approaches to online identity based on *blockchain* distributed ledger technology. Distributed
ledgers are a distributed systems design pattern that yields a system that works like an append-only logbook.
Participants can create new entries in this logbook, but no one—neither the original author, nor other participants—can
retroactively change a logbook entry once it has been written. In the blockchain model, past entries are essentially
written into stone. This near-perfect immutability is what opens them for a number of use cases from cryptographic
pseudo-currencies [#cryptocurrency]_.
An overview over a variety of these unconventional blockchain identity verification approaches can be found in `this
unpublished 2020 survey by Siddarth, Ivliev, Siri and Berman <https://arxiv.org/ftp/arxiv/papers/2008/2008.05300.pdf>`_.
They walk their readers through a number of different projects that try to solve the question "Is this human who they
pretend to be?" using joint socio-technological approaches. In the following few sections, you may find a short outline
of a small selection of them. The conlusion of this post will be a commentary on these approaches, and on the underlying
problem of identity in a digital world.
.. BrightID
In one scheme, identity is determined by "notary" computers that aggregate large amounts of information on a user's
social contacts. These computers then run an algorithm derived from the SybilGuard_, SybilLimit_ and SybilInfer_ lineage
of random-walk based algorithms. These algorithms assume that authentic social graphs are small world graphs: Everyone
knows everyone else through a friend's friend's friend. They also assume that there is an upper bound on how many
connections with authentic users an attacker can forge: Anyone who is not embedded into the graph well enough is cut
out. Like this, they put an upper limit on the number of theia identites an attacker can assume given a certian number
of connections to real people.
Disregarding the catastrophic privacy issues of storing large amounts of data on social relationships on someone else's
computer, this second assumption is where this model unfortunately breaks down. Applying common sense, it is completely
realistic for an attacker to forge a large number of social connections: This is precisely what most of social media
marketing is about! A more malicious angle on this would be to consider how in meatspace [#meatspacefn]_ multi-level
marketing schemes are successful in coaxing people to abuse their social graphs to disastrous consequences to the
well-being of themselves and others. Similar schemes would certainly be possible in cyberspace as well. An additional
point to consider is that the upper limit SybilGuard_ and others place on the number of fake identities one can have is
simply not that strict at all. An attacker could still get away with a reasonable number of false identities before
getting caught by any such algorithm.
.. Duniter
In another scheme, identity is awarded to anyone who can convince several people already in the network to vouch for
them, and who is at most a few degrees removed from one of several pre-determined celebrities. Apart from again being
vulnerable to conmen and other scammers, this system has the glaring flaw of roundly refusing to recognize any person
who is not willing or able to engage with multiple of its members. Along with the system's informal requirement for
members to only vouch for people they have physically met this leads to a nonstarter in a cyberspace that grown
specifically *because* it transcends national borders and physical distance—two most serious obstacles to in-person
communication.
.. Idena Network
The last scheme I will outline in this post is based around a set of `Turing tests`_; that is, quizzes that are designed
to tell apart man and machine. In this system, all participants have to simultaneously undergo a Turing test once in a
fortnight. The idea is that this limits the number of theia identities an attacker can assume since they can only solve
that many Turing tests at the same time. The system uses a particular type of picture classification-based Turing test
and does not seem to be designed with the blind or mentally disabled in mind with accessibility concerns nowhere to be
found in the so-called "manifesto" published by its creators. But even ignoring that, the system obviously fails at an
even more basic level: The idea that everyone takes a Turing test at the same time only works in a world without time
zones. Or jobs for that matter. Also, it assumes that an attacker cannot simply hire a small army of people someplace
else to fool the system.
.. _SybilLimit: https://www.comp.nus.edu.sg/~yuhf/yuh-sybillimit.pdf
.. _SybilGuard: http://www.math.cmu.edu/~adf/research/SybilGuard.pdf
.. _SybilInfer: https://www.princeton.edu/~pmittal/publications/sybilinfer-ndss09.pdf
.. _`Turing Tests`: https://en.wikipedia.org/wiki/Turing_test
Identity between Cyberspace and Meatspace
=========================================
A common thread in these solutions, from the Facebook'esque Stasi_ methods to the crypto-anarchist challenge-response
utopias, is that they all approach digital identity as a question of Objective Truth™ that can unanimously be decided at
a system level—or that can be externalized to the next larger system such as the state. Alas, the important question
remains unasked:
What *is* identity?
The answer to this question certainly depends on the system being examined. For example, an important reason the
capitalist corporations mentioned above require knowledge about their users' identity is to generate plausible
statistics for the advertisers that form their customer base, similar to how a farmer will keep statics on yield and
quality for the buyers of his crop. With this background, a full decoupling of platform accounts from a notion of legal
identity seems at odds with the platform's business model—and we will have to adjust our expectations for reform
accordingly.
A common thread among all systems mentioned above is that they all have a social component to them. For this common use
case of social systems, I want to make a suggestion on how we can approach digital identity in a more practical, less
discriminatory [#discriminatory]_ manner than any of the methods we discussed above. I think both using people's social
connections and proxying the decisions of external authorities such as the state are bad systems to decide who is a
person and who is not. I will now illustrate this point a bit. Let us think about how many digital identities a human
beign might have. First, consider the case of n=0, someone who simply wants no business with the system at all. For
simplicity, let us assume that we have solved this issue of consent, i.e. every person who is identified by the system
consents to this practice. For n=1, the approaches outlined above all provide some approximate solution. States may not
grant every human sufficient ID (e.g. children, the mentally disabled or prisoners might be left out), and the social
systems might fail to catch people who simply do not have any friends, but otherwise their approximations hold. Maybe.
But what about n=2, n=3, ...? None of these systems adequately consider cases where a human being might legitimately
wish to hold multiple digital identities, non-maliciously.
Consider a hypothetical lesbian, conservative politician. An active social media presence is a core component of a
modern politician's carreer. At the same time, "conservative homophobe" is still well within the realm of tautology and
it would be legitimate for this politician to wish to not disclose a large fraction of their private life to the world
at large. They might have a separate online identity for matters related to it. For this politician, the social
relationship-based systems referenced above would either incorporate outing as a design feature, or they would force
the politician to choose either of their two identities: To choose between private life and carreer. When deferring to
the state as the decider over personhood, at least the platform's operator would know about the outrageously sensitive
link between the politician's online identities. Clearly, no such solution can be considered socially just.
Let us try not to be caught up on saving the world at this point. The issue of conservative homophobia is out of the
scope of our consideration, and it is not one that anyone can solve in the near future. Magical realism aside, least of
all can some technological thing beckon this change. There is a case for legitimate uses of multiple, separate digital
identities, and we do not have a technical or political answer to it. All hope is not lost yet, though. We can easily
undo this gordian knot by acknowledging an unspoken assumption that underlies any social relationships between real
people, past the procrustean bed of computer systems or organizational structures these relationships are cast into.
As a function of social interaction, digital identities conform to roles_ in sociological terminology, and are not
at all the same as personhood_. Roles are subjective and arise from a relationship between people, and a single
person might legitimately perform different roles depending on context.
When computer scientists or programmers are creating new systems, there always is an (often implicit) modelling stage.
Formally, during this stage a domain expert and a modeller with a computer science background come together, each
contributing their knowledge to form a model that is both appropriate for real-world use and practical from an
engineering point of view. In practice, these two roles are often necessarily fulfilled by the same person, who is often
also the programmer of the thing. This leads to many computer systems using poor models. A typical example of this issue
are systems requiring a person's name that use three input fields labelled "First Name", "Middle Initial" and "Last
Name". These systems are often created by US-American programmers, who are used to this naming schema from their lived
experience. Unfortunately, this schema breaks down for those few billion people who use their last name first, who have
more than one middle name, or who have multiple given names and do not normally use the first one of those.
Once a system creator's implicit assumptions have been encoded into the system like this, it is often very hard to get
out of that situation. A pattern to use during careful modelling is to keep the model flexible to account for unforeseen
corner cases. For example, when modelling a system requiring a person's name, one would have to ask what the name is
used for. It may be the most sensible decision to simply ask the user for their name twice: Once in first name/last name
format for e.g. tax purposes, and once with a free-form text field for e.g. displaying on their account page.
While for names, many systems already use some form of flexible model by e.g. having a *handle* or *nickname* separate
from the *display name*, "social" systems still often are stuck with an identity model based around a concept of a
single, rigid identity. In practice, people perform different roles_ in different circumstances. When asking for a
person's identity, one would get wildly different answers from different people. A person's identity as perceived by
others is coupled to their relationship more than to some underlying, biological or administrative truth. Thinking back
to the straw man politician above, this is evident in subtle ways in almost all our everyday relationships: Some people
may know me by my legal name, some by my online nickname. To some I may be a computer scientist, to some a flatmate.
None of my friends and acquaintances have ever wanted to see my passport, or asked to take my DNA to ascertain that I am
a distinct human being from the other humans they know. Likewise, identifying me by my social connections is impractical
as it would require an exceedingly weird amount of what can only be described as snooping. Yet, this concept of a
single, consistent, global, true identity is exactly what up to now all technological solutions to the identity problem
are trying to achieve.
Building Bridges
================
I think I can offer you one main take-aways from the discussion above.
During modelling social systems, focus on relationships—not identity.
Rephrased into more actionable points, as someone designing a social digital system, do the following:
0. Early in the design stages, take the time to consider fundamental modelling issues like this one. If you don't, you
will likely get stuck with a sub-optimal model that will be hard to get rid of.
1. Where possible, be flexible. Allow people to chose their own identifier. Don't require them to use their real names,
they may not wish to disclose those or they may not be in a format that is useful to you (they may be too long, too
short, too ubiquituous, in foreign characters etc.). A free-form text field with a reasonable length limit is a good
approach here.
2. Do not use credit cards or phone numbers to identify people. There are many people who do not have either, and
scammers can simply buy this data in bulk on the darknet.
3. Allow people to create multiple identites [#accountswitchopsec]_, and acknowledge the role of social relationships in
your interaction features. People have very legitimate reasons to separate areas of their lifes, and it is not for
you or your computer to decide who is who to whom. If your thing requires a global search function, re-consider the
data protection aspects of your system. If you want to encourage social functions in the face of bots and trolls,
make it easy for people to share their identities out-of-band, such as through a QR code or a copy-and-pasteable
short link. If you require someone's legal name or address for billing purposes, unify these identities behind the
scenes if at all and allow them to act as if fully independent in public.
While change of perspective comes with its share of user experience challenges, but also with a promise for a more
human, more dignified online experience. Perhaps we can find a way to adapt cyberspace to humans, instead of continuing
trying it the other way around.
.. _astroturfing: https://en.wikipedia.org/wiki/Astroturfing
.. _Stasi: https://en.wikipedia.org/wiki/Stasi
.. [#cryptocurrency] Pseudo-currencies in that, while they provide some aspects of a regular currency such as ownership
and transactions, they lack most others. Traditional currencies are backed by states, regulated by central banks
tasked with maintaining their stability and ultimately provide accountability through law enforcement, courts
and political elections.
.. [#discriminatory] Discriminatory as in discriminating against minorities, but also as in deciding what is and what is
not.
.. [#accountswitchopsec] This does mean that you should not actively prevent people from creating multiple accounts. It
does not necessarily entail building a proper user interface around this practice. If you do the latter, e.g. by
offering a "switch identity" button or an identiy drop-down menu on a post submission form, you can easily
encourage slip-ups that might disclose the connection between two identities, and you make it possible for
someone hacking a single login to learn about this connection as well.
.. [#meatspacefn] Meatspace_ is where people physically are, as opposed to cyberspace
.. _Meatspace: https://dictionary.cambridge.org/dictionary/english/meatspace
.. _roles: https://en.wikipedia.org/wiki/Role
.. _personhood: https://en.wikipedia.org/wiki/Personhood

89
content/posts/sybil-resistance-identity/index.rst
View file

@ -1,89 +0,0 @@
---
title: "Identity between Cyberspace and Meatspace"
date: 2020-09-09T15:00:00+02:00
draft: true
---
.. raw:: html
<figure class="header">
<img src="images/succulents.jpg">
<figcaption>Photo by <a href="https://unsplash.com/@timbennettcreative">Tim Bennett</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption>
</figure>
Identity in Cyberspace
======================
.. Identity is a frequent problem
.. Easy solutions abound
.. Precise modelling is uncommon
.. True identity is sensitive, hard to handle
..
.. Often, conversational features emphasized -> true identity is unnecessary
.. Social role theory
.. Call to action
Most computer systems that interface with humans have a concept of user identity. The data structures used for its
storage vary, but usually one *account* corresponds to one human *user*. In many applications, the system operator tries
to ensure that one user cannot create multiple accounts. In online social networks, astrotufing_ and trolling are easier
to fight when limits are imposed on account creation. In online stores, fraud prevention means the store operator needs
their customers legal identity and the operator must be able to ban offending customers. In mobile messaging systems,
users have to be able to find each other by some identifier such as name or phone number, and this identifier has to be
unique and hard to forge.
Today, in systems that allow anyone to create an account have largely converged to require either an email address or a
mobile phone number. Email addresses are used by systems that are less vulnerable to abuse and that are used on laptop
or desktop computers. Mobile phone numbers are abundantly used in smartphone apps, as well as in systems more prone to
abuse such as online social networks or ecommerce. Both are easily verified using a confirmation email or SMS.
When designing or programming an online system, it is uncommon that the precise real-world semantics of accounts are
modelled. Most computer systems use ad-hoc data models. During their creation, their programmers implicit assumptions
about the world are encoded into these data models. Most of the time this works fine, but it does lead to significant
blind spots that can make systems break down for a fraction of their users.
Lives in Meatspace
==================
A consequence of the proliferation of phone numbers being used to identify people is that most people will not be able
to create multiple accounts. *"That's the point!"* you might say, but while we want to prevent scammers, spammers and
boored schoolchildren from messing with our systems, everybody else may have legitimate reasons to have more than one
account.
We can apply sociology's model of roles_ to understand this issue. In sociology, a role is the comprehensive pattern of
rules and expectations that govern an individual's behavior corresponding to their social position. A key fact is that
most people occupy mutliple roles. A parent may also be a company employee or a wife and perform accordingly given the
circumstances. Systems that tie digital identity to legal personhood through the contracts behind phone numbers impede
their users' attempts at role separation. Effects of this are e.g. that nowadays employers routinely screen applicants'
social media accounts for unacceptable content.
While this role conflict merely amounts to a minor inconvenience to most there are many to who it poses an existential
problem. Consider an LGBT+ person living in a repressive country or a politically conservative person living in a
very liberal city. Both have legitimate reasons to strictly separate parts of their private lives from others. For both,
much is at stake. Yet, both will have to practically circumvent most online systems registration barriers to implement
this separation.
Trusting the User
=================
While there is no single solution to these issues, there are several possible mitigations. The first and most important
one is to systematically think about the system's data model when creating it. Which assumptions about the real world
are inherent in it? Are these assumptions likely to cause issues? Ad-hoc models are easily created, but hard to get rid
of when they start causing problems.
A general guideline on identity should be that hindering trolls by requiring things like phone numbers or credit card
numbers is very likely to also be an obstacle to many entirely legitimate uses. Captchas_ or invitation links can help
to keep out the trolls. Another approach is to limit the damage a troll can cause with things like effective moderation
systems, reputation systems or by limiting the reach of newly created accounts.
Outside of e-commerce, actually tying a digital account to a real-world identity is very rarely necessary. The value of
a messenger app is not in the names in its contacts list, but the conversations behind these names. When two people meet
each other on the street, their interaction is shaped by a myriad of social factors—but *not* by them showing each other
their photo ID.
Humans with their messy identities do not fit today's cyberspace well. Let's adapt cyberspace to humans, instead of
trying it the other way around.
.. _astroturfing: https://en.wikipedia.org/wiki/Astroturfing
.. _roles: https://en.wikipedia.org/wiki/Role
.. _Captchas: https://link.springer.com/content/pdf/10.1007/3-540-39200-9_18.pdf

60
content/posts/thors-hammer/index.rst
View file

@ -1,60 +0,0 @@
---
title: "Thor's Hammer"
date: 2018-05-03T11:59:37+02:00
---
In case you were having an inferiority complex because your friends' IBM Model M keyboards are so much louder than the
shitty rubber dome freebie you got with your pc... Here's the solution: Thor's Hammer, a simple typing cadence enhancer
for `PS/2`_ keyboards.
.. raw:: html
<figure>
<video controls loop>
<source src="video/thors_hammer.mov" type="video/h264">
<source src="video/thors_hammer.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>A demonstration of the completed project.
<a href="video/thors_hammer.mov">h264 download</a> /
<a href="video/thors_hammer.webm">webm download</a>
</figcaption>
</figure>
The connects to the keyboard's PS/2 clock line and briefly actuates a large solenoid on each key press. An interesting
fact about PS/2 is that the clock line is only active as long as either the host computer or the input device actually
want to send data. In case of a keyboard that's the case when a key is pressed or when the host changes the keyboard's
LED state, otherwise the clock line is silent. We ignore the LED activity for now as it's generally coupled to key
presses. By just triggering an NE555 configured as astable flipflop we can stretch each train of clock pulses to a
pulse a few tens of milliseconds long that is enough to actuate the solenoid.
.. raw:: html
<figure>
<img src="images/thors_hammer_schematic.jpg" alt="The schematic of the PS2 driver">
<figcaption>The schematic of the driver stretching the PS/2 clock pulses to drive the solenoid.</figcaption>
</figure>
Since PS/2 sends each key press and key release separately this circuit will pulse twice per keystroke. It would be
possible to ignore one of them but I figure the added noise just adds to the experience.
Built on a breadboard, the circuit looks like this.
.. raw:: html
<figure>
<img src="images/thors_hammer_breadboard.jpg" alt="The circuit built on a breadboard">
<figcaption>The completed circuit built up on a breadboard and attached to a keyboard.</figcaption>
</figure>
Since my solenoid did not have a tensioning spring I used a rubber band and some vinyl tape to make an adjustable
tensioner. The small orange USB hub serves as an end-stop because I had nothing else of the right shape. The sound and
resonance of the thing can be adjusted to taste by moving the end stop, adjusting the tensioning rubber and tuning the
excitation duration using the potentiometer. My particular solenoid was a bit slow so I added some pieces of circuit
board as shims between the plunger and the case to limit the plunger's travel inside the solenoid core.
.. _`PS/2`: https://en.wikipedia.org/wiki/PS/2_port

162
content/posts/wifi-led-driver/index.rst
View file

@ -1,162 +0,0 @@
---
title: "Wifi Led Driver"
date: 2018-05-02T11:31:03+02:00
---
Project motivation
==================
.. FIXME finished project picture with LED tape
.. raw:: html
<figure>
<img src="images/lyza_board_in_case.small.jpg">
<figcaption>The completed driver board installed in the 3D-printed case. This device can now be connected to
12V and two segments of LED tape that can then be controlled trough Wifi. The ESP8266 module goes on the pin
header on the left and was removed for this picture.
</figcaption>
</figure>
After the `multichannel LED driver`_ was completed, I was just getting used to controlling LEDs at 14-bit resolution.
I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases. Sometimes I
just want to pop a piece of LED tape or two somewhere, but I don't need a full 32 channels of control. I ended up
thinking that a smaller version of the 32-channel driver that didn't require a separate control computer would be
handy. So I sat down and designed a variant of the design with only 8 channels instead of 32 and an on-board ESP8266_
module instead of the RS485_ transceiver for WiFi connectivity.
The Electronics
===============
The schematic was mostly copy-pasted from the 32-channel design. The PCB was designed from scratch. This time, I went
for a 5x7cm form factor to allow for enough room for all connectors and to give the ESP8266_'s WiFi antenna enough
space. The board has two 5-pin Phoenix-style_ for two RGB-White (RGBW) tapes and one 2-pin Phoenix-style_ connector for
12V power input. The control circuitry and the serial protocol are unchanged, but the STM32_ now talks to an ESP-01_
module running custom firmware.
The LEDs are driven using a 74HC595_ shift register controlling a bunch of AO3400_ MOSFETs_, with resistors in front of
the MOSFETs_' gates to slow down the transitions a bit to reduce brighntess nonlinearities and EMI_ resulting from
ringing of the LED tape's wiring inductance.
The board has two spots for either `self-resettable fuses (polyfuses) <polyfuse_>`__ or regular melting-wire fuses_ in
a small SMD_ package, one for each RGBW output. For low currents the self-resettable fuses should be okay but at higher
currents their `trip times get long enough that they become unlikely to trip in time to save anything
<littlefuse-16r-datasheet_>`__, so plain old non-resettable fuses would be the way to go there.
.. FIXME finished board photos
.. FIXME board with test tape picture
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/schematic.png">
<figcaption>
The schematic of the driver board, with the ESP8266 on the top left, the STM32 microcontroller for LED
modulation below, the shift register in the middle and the LED drivers and outputs on the right.
<a href="resource/lyza_schematic_and_pcb.pdf">Download PDF</a>
</figcaption>
</figure><figure class="side-by-side">
<img src="images/layout.png">
<figcaption>
The board layout with the top side being visible. The top side contains the footprint for the ESP8266, the
microcontroller, fuses, filter cap, connectors and the shift register. The LEDs are connected on the left,
with one connector per LED tape segment. The power input connector is on the bottom right. The LED driver
MOSFETs are in small SOT-23 packages on the back of the board. Since this board is not intended for
super-high currents, the MOSFETs are adequately cooled just through the board's copper planes.
<a href="resource/lyza_schematic_and_pcb.pdf">Download PDF</a>
</figcaption>
</figure>
</figure>
.. raw:: html
<figure>
<img src="images/lyza_boards.small.jpg">
<figcaption>The completed PCBs of this project (front) and the `multichannel LED driver`_ project the driver
circuitry was derived from (back).
</figcaption>
</figure>
The Firmware
============
The STM32_ firmware only had to be slightly modified to accomodate the reduced channel count since the protocol remains
unchanged. The ESP firmware is based on esphttpd_ by Spritetm_. The modifications to the webserver firmware are pretty
basic. First, the UART console has been disabled since I use the UART to talk to the STM32. The few bootloader messages
popping out the UART on boot are not an issue, since they're unlikely to contain the fixed 32-bit address prefix the
serial protocol requires for the STM32_ to do anything.
Second, I added LED control by adding drivers for the serial protocol and a bunch of colorspace conversion functions.
When I first tested the prototype software, I noticed that color reproduction was extremely poor. When I just sent a
HSV_ rainbow fade from a python command line, the result looked totally wrong. The fade did not seem to go at a constant
speed and some colors, in particular yellow, orange and greens, were not visible at all. The problem turned out to be a
stark mismatch of the red, green and blue channels of the LED tape and less-than-optimal color reproduction of the pure
colors. I decided to properly measure the LED tape's color reproduction so I could compensate for it in software. This
turned out to be an extremely interesting project, the details of which you can read in my `LED characterization`_
article.
Third, I updated the built-in websites with some ad-hoc documentation on how to use the thing and a basic interface for
LED control.
.. FIXME screenshot of firmware website
Making an enclosure
===================
To be actually useful, the driver needed a robust enclosure. Bare PCBs are nice for prototyping, but for actually
putting the thing anywhere it needs a case to protect it against random destruction.
The board has four mounting holes with comfortable spacing in its corners to allow easy mounting inside a 3D-printed
case. The case itself is described in an OpenSCAD_ script. To make it look a little nicer, a little 3D relief is laid
into the lid. The 3D relief is generated with a bit of blender magic. The source STL_ model is loaded into blender, then
blender's amazingly flexible rendering system is used to export a depth map of a projection of the model as a PNG_ file.
This depth map is then imported as a triangle mesh into OpenSCAD_.
For the relief to look good, I chose a rather high resolution for the depth map. This unfortunately leads to extreme
memory use and processing time on the part of OpenSCAD_, but since I have access to a sufficiently fast machine that is
not a problem. Just be careful if you try opening the OpenSCAD_ file on your machine, OpenSCAD_ will probably crash
unless you're on a beefy machine or interrupt it when it starts auto-rendering the file.
The board is mounted into the enclosure using knurled insert nuts that are pressed into a 3D-printed hole using a bit of
violence.
.. FIXME openscad screenshot
.. FIXME enclosure parts
.. FIXME finished enclosure with board inside
.. raw:: html
<figure>
<figure class="side-by-side">
<img src="images/lyza_case_raw.small.jpg">
<figcaption>The 3D-printed case with threaded inserts before painting.</figcaption>
</figure><figure class="side-by-side">
<img src="images/lyza_case_complete.small.jpg">
<figcaption>The 3D-printed case with the board installed after painting. This was my first attempt at
painting a 3D-printed case so it looks pretty bad.</figcaption>
</figure>
</figure>
.. _`multichannel LED driver`: {{<ref "posts/multichannel-led-driver/index.rst">}}
.. _`LED characterization`: {{<ref "posts/led-characterization/index.rst">}}
.. _ESP8266: https://en.wikipedia.org/wiki/ESP8266
.. _RS485: https://en.wikipedia.org/wiki/RS-485
.. _Phoenix-style: https://www.phoenixcontact.com/online/portal/de?uri=pxc-oc-itemdetail:pid=1757019&library=dede&tab=1
.. _STM32: http://www.st.com/resource/en/datasheet/stm32f030f4.pdf
.. _ESP-01: http://www.watterott.com/de/ESP8266-WiFi-Serial-Transceiver-Modul
.. _74HC595: http://www.ti.com/lit/ds/symlink/sn74hc595.pdf
.. _AO3400: http://aosmd.com/pdfs/datasheet/AO3400.pdf
.. _MOSFETs: https://en.wikipedia.org/wiki/MOSFET
.. _EMI: https://en.wikipedia.org/wiki/Electromagnetic_interference
.. _polyfuse: https://en.wikipedia.org/wiki/Resettable_fuse
.. _SMD: https://en.wikipedia.org/wiki/Surface-mount_technology
.. _fuses: https://en.wikipedia.org/wiki/Fuse_(electrical)
.. _littlefuse-16r-datasheet: http://m.littelfuse.com/~/media/electronics/datasheets/resettable_ptcs/littelfuse_ptc_16r_datasheet.pdf.pdf
.. _OpenSCAD: http://www.openscad.org/
.. _STL: https://en.wikipedia.org/wiki/STL_(file_format)
.. _PNG: https://en.wikipedia.org/wiki/Portable_Network_Graphics
.. _esphttpd: https://github.com/Spritetm/esphttpd
.. _Spritetm: http://spritesmods.com/
.. _`HSV`: https://en.wikipedia.org/wiki/HSL_and_HSV

0
themes/hugo-classic/static/css/fonts.css → css/fonts.css
View file

0
themes/hugo-classic/static/css/style.css → css/style.css
View file

25
deploy.py
View file

@ -1,25 +0,0 @@
#!/usr/bin/env python3
import tempfile
import subprocess
from pathlib import Path
if __name__ == '__main__':
with tempfile.TemporaryDirectory() as tmpdir:
current_branch = subprocess.run(['git', 'symbolic-ref', '-q', 'HEAD'], check=False, capture_output=True).stdout.strip()
if current_branch == 'refs/heads/deploy':
raise SystemError('This script cannot be run from the deploy branch (run from main instead)')
subprocess.run(['git', 'worktree', 'add', '--detach', tmpdir], check=True)
try:
subprocess.run(['hugo'], cwd=tmpdir, check=True)
subprocess.run(['git', 'add', '--force', 'public'], cwd=tmpdir, check=True)
write_tree = subprocess.run(['git', 'write-tree', '--prefix=public/'], cwd=tmpdir, check=True, capture_output=True)
tree = write_tree.stdout.strip()
commit = subprocess.run(['git', 'commit-tree', '-p', 'HEAD', '-p', 'refs/heads/deploy', '-m', 'deploy.py auto-commit', tree], cwd=tmpdir, check=True, capture_output=True).stdout.strip()
subprocess.run(['git', 'update-ref', '-m', f'deploy.sh update deploy branch to {commit}', 'refs/heads/deploy', commit], cwd=tmpdir, check=True)
finally:
subprocess.run(['git', 'worktree', 'remove', '-f', tmpdir], check=True)

173
imprint/index.html Normal file
View file

@ -0,0 +1,173 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Impressum | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<div class="article-meta">
<h1><span class="title">Impressum</span></h1>
<p class="terms">
</p>
</div>
<main>
<div class="document">
<div class="section" id="impressum">
<h2>Impressum</h2>
<p>
Sebastian Götte<br/>
c/o Praxis Dr. Götte<br/>
Krankenhausstr. 1a<br/>
54634 Bitburg<br/>
imprint@jaseg.net
</p>
Inhaltlich Verantwortlicher gem. § 55 II RStV: Sebastian Götte (Anschrift s.o.)</div>
<div class="section" id="lizenz-dieser-seite">
<h2>Lizenz dieser Seite</h2>
Dieses Werk ist lizenziert unter einer
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">
Creative Commons Namensnennung - Nicht-kommerziell - Weitergabe unter gleichen Bedingungen 4.0 International
Lizenz (CC-BY-NC-SA)
</a>.<br/>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">
<img alt="Creative Commons Lizenzvertrag" style="border-width:0"
src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" />
</a></div>
<div class="section" id="haftungsbeschrankung-fur-inhalte-der-website">
<h2>Haftungsbeschränkung für Inhalte der Website</h2>
<p>Gemäß § 7 Abs. 1 TMG ist der Verantwortliche der Website i. S. v. § 5 TMG für eigene Informationen, die er zur Nutzung
bereithält, nach den allgemeinen Gesetzen verantwortlich.</p>
<p>Der Verantwortliche der vorbezeichneten Website übernimmt keine Haftung auf Aktualität, Richtigkeit und Vollständigkeit
der auf dieser Website zur Verfügung gestellten Inhalte. Dies gilt nicht, wenn dem Verantwortlichen vorsätzliches oder
grob fahrlässiges Verhalten vorzuwerfen ist. Die Inhalte wurden mit der größtmöglichen Sorgfalt erstellt. Dennoch kann
die inhaltliche Richtigkeit insbesondere bei komplexen Themen nicht gewährleistet werden, so dass der Verantwortliche
den Nutzern empfiehlt, bei wichtigen Informationen bei den zuständigen Stellen anzufragen oder rechtliche Beratung in
Anspruch zu nehmen. Sofern kostenpflichtige Inhalte oder Dienste auf der Website zur Verfügung gestellt werden, handelt
es sich dabei um unverbindliche Invitatio ad offerendum, welche lediglich zur Abgabe eines Angebots durch den Nutzer
aufrufen und selbst kein verbindliches Angebot darstellen.</p>
<p>Gemäß §§ 8 ff. TMG ist der Website-Betreiber für fremde Inhalte, die er für einen Nutzer veröffentlicht, nicht
verantwortlich, sofern</p>
<ul class="simple">
<li>er keine Kenntnis von der rechtswidrigen Handlung oder der Information hat und ihm im Falle von
Schadensersatzansprüchen auch keine Tatsachen oder Umstände bekannt sind, aus denen die rechtswidrige Handlung oder
die Information offensichtlich wird, oder</li>
<li>er unverzüglich tätig geworden ist, um die Information zu entfernen oder den Zugang zu ihr zu sperren, sobald er
diese Kenntnis erlangt hat.</li>
</ul>
<p>Erlangt der Website-Betreiber Kenntnis von solchen rechtswidrigen Inhalten, werden diese unverzüglich entfernt.</p>
</div>
<div class="section" id="haftung-fur-ausgehende-links">
<h2>Haftung für ausgehende Links</h2>
<p>Der Website-Betreiber verlinkt von seiner Website auf fremde Websites. Durch diese sog. „Hyperlinks“ wird der Nutzer
direkt auf die fremde Website geleitet. Der Website-Betreiber hat dabei keinerlei Einfluss auf die Informationen der
fremden Website. Daher kann keine Haftung für die Aktualität, Richtigkeit und Vollständigkeit der Inhalte der fremden
Website übernommen werden. Der Website-Betreiber versichert jedoch, dass ihm zum Zeitpunkt des Setzens der Verlinkung
keinerlei rechtliche Verstöße bekannt waren und er die fremde Website im Rahmen des Zumutbaren geprüft hat.</p>
<p>Erhält der Website-Betreiber Kenntnis von der Rechtswidrigkeit der verlinkten Inhalte, wird der entsprechende Link
unverzüglich entfernt.</p>
</div>
<div class="section" id="urheberrechte">
<h2>Urheberrechte</h2>
<p>Die auf dieser Webseite veröffentlichten Inhalte unterliegen dem deutschen Urheberrecht.</p>
<p>Als Urheber i. S. v. § 7 UrhG stehen dem Website-Betreiber die alleinigen und ausschließlichen Verwertungsrechte gemäß
§§ 15 ff. UrhG zu. Eine Vervielfältigung oder Verwendung sämtlicher Inhalte der Website in fremden elektronischen oder
gedruckten Medien ist ohne vorherige Zustimmung des Website-Betreibers untersagt und wird ggf. verfolgt.</p>
</div>
<div class="section" id="datenschutz">
<h2>Datenschutz</h2>
<p>Die Website kann grundsätzlich ohne Eingabe von personenbezogenen Daten wie z. B. Name oder E-Mail-Adresse genutzt
werden. Sollte die Möglichkeit der Eingabe solcher Daten bestehen, so erfolgt die Mitteilung dieser Daten auf
freiwilliger Basis durch den Nutzer. Eine Weitergabe dieser Daten an Dritte ist ohne die ausdrückliche Zustimmung durch
den Nutzer ausgeschlossen.</p>
<p>Ein lückenloser Schutz der übermittelten Daten vor dem Zugriff durch unbefugte Dritte ist jedoch nicht möglich, da es im
Bereich der Datenübermittlung im Internet zu Sicherheitslücken kommen kann. Der Website-Betreiber versucht jedoch, die
Gefahr des unbefugten Zugriffs durch geeignete Maßnahmen zu unterbinden und im Falle der Kenntnis von einer
Sicherheitslücke diese durch geeignete Maßnahmen zu schließen.</p>
<p>Der Nutzung der im Impressum veröffentlichten Kontaktdaten durch Dritte zur Übersendung von nicht ausdrücklich
gewünschter Werbung, insbesondere Spam-Mails, wird widersprochen und im Falle der Nichtbeachtung ggf. mit rechtlichen
Schritten verfolgt.</p>
<p>(Quelle: <a class="reference external" href="https://www.kanzlei-wirtschaftsrecht.berlin/aktuelles/kostenloser-muster-disclaimer">BUSE HERZ GRUNST Rechtsanwälte</a>)</p>
</div>
</div>
</main>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

89
index.html Normal file
View file

@ -0,0 +1,89 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta name="generator" content="Hugo 0.87.0" />
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>jaseg.de | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left current">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<ul>
<li>
<span class="date">2020/08/18</span>
<a href="/posts/">Posts</a>
</li>
</ul>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

105
index.xml Normal file
View file

@ -0,0 +1,105 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>jaseg.de</title>
<link>https://blog.jaseg.de/</link>
<description>Recent content on jaseg.de</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<lastBuildDate>Tue, 18 Aug 2020 13:15:39 +0200</lastBuildDate><atom:link href="https://blog.jaseg.de/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Kicad Mesh Plugin</title>
<link>https://blog.jaseg.de/posts/kicad-mesh-plugin/</link>
<pubDate>Tue, 18 Aug 2020 13:15:39 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/kicad-mesh-plugin/</guid>
<description>Tamper Detection Meshes Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for groceries with a credit card, messaging a friend on Signal or unlocking a car with its keyfob would not be possible.</description>
</item>
<item>
<title>Private Contact Discovery</title>
<link>https://blog.jaseg.de/posts/private-contact-discovery/</link>
<pubDate>Sat, 22 Jun 2019 10:30:00 +0800</pubDate>
<guid>https://blog.jaseg.de/posts/private-contact-discovery/</guid>
<description>Private Contact Discovery Private Contact Discovery (PCD) is the formal name for the problem modern smartphone messenger applications have on installation: Given a user&#39;s address book, find out which of their contacts also use the same messenger without the messenger&#39;s servers learning anything about the user&#39;s address book. The widespread non-private way to do this is to simply upload the user&#39;s address book to the app&#39;s operator&#39;s servers and do an SQL JOIN keyed on the phone number field against the database of registered users.</description>
</item>
<item>
<title>Hardware Security Module Basics</title>
<link>https://blog.jaseg.de/posts/hsm-basics/</link>
<pubDate>Fri, 17 May 2019 15:29:20 +0800</pubDate>
<guid>https://blog.jaseg.de/posts/hsm-basics/</guid>
<description>Hardware Security Modules and Security Research and Cryptography On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of Prof. Mori&#39;s security research working group at Waseda University. The motivation for this was that outside of low-level hardware security people and people working in the financial industry HSMs are not thought about that often. In particular most network or systems security people would not consider them an option.</description>
</item>
<item>
<title>How to talk to your microcontroller over serial</title>
<link>https://blog.jaseg.de/posts/serial-protocols/</link>
<pubDate>Sat, 19 May 2018 08:09:46 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/serial-protocols/</guid>
<description>Scroll to the end for the TL;DR.
In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error conditions.
If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to another microcontroller or to a computer.</description>
</item>
<item>
<title>Thor&#39;s Hammer</title>
<link>https://blog.jaseg.de/posts/thors-hammer/</link>
<pubDate>Thu, 03 May 2018 11:59:37 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/thors-hammer/</guid>
<description>In case you were having an inferiority complex because your friends&#39; IBM Model M keyboards are so much louder than the shitty rubber dome freebie you got with your pc... Here&#39;s the solution: Thor&#39;s Hammer, a simple typing cadence enhancer for PS/2 keyboards.
Your browser does not support the HTML5 video tag. A demonstration of the completed project. h264 download / webm download The connects to the keyboard&#39;s PS/2 clock line and briefly actuates a large solenoid on each key press.</description>
</item>
<item>
<title>32-Channel LED tape driver</title>
<link>https://blog.jaseg.de/posts/multichannel-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:14 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/multichannel-led-driver/</guid>
<description>Theoretical basics Together, a friend and I outfitted the small staircase at Berlin&#39;s Chaos Computer Club with nice, shiny RGB-WW LED tape for ambient lighting. This tape is like regular RGB tape but with an additional warm white channel, which makes for much more natural pastels and whites. There are several variants of RGBW tape. Cheap ones have separate RGB and white LEDs, which is fine for indirect lighting but does not work for direct lighting.</description>
</item>
<item>
<title>Wifi Led Driver</title>
<link>https://blog.jaseg.de/posts/wifi-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:03 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/wifi-led-driver/</guid>
<description>Project motivation The completed driver board installed in the 3D-printed case. This device can now be connected to 12V and two segments of LED tape that can then be controlled trough Wifi. The ESP8266 module goes on the pin header on the left and was removed for this picture. After the multichannel LED driver was completed, I was just getting used to controlling LEDs at 14-bit resolution. I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases.</description>
</item>
<item>
<title>LED Characterization</title>
<link>https://blog.jaseg.de/posts/led-characterization/</link>
<pubDate>Wed, 02 May 2018 11:18:38 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/led-characterization/</guid>
<description>Preface Recently, I have been working on a small driver for ambient lighting using 12V LED strips like you can get inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when fading between different colors, the colors look all wrong!</description>
</item>
<item>
<title>About jaseg</title>
<link>https://blog.jaseg.de/about/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://blog.jaseg.de/about/</guid>
<description>About Hej, I&#39;m Jan, or jaseg. At the moment I&#39;m doing a PhD (Dr.-Ing.) at TU Darmstadt in Computer Science, specializing on Hardware Security. This is my personal website where I publish things that I find interesting.
I self-host my code at git.jaseg.de, but I am also on github and on gitlab. I use github for issue tracking for some of my projects such as gerbolyze and python-mpv. I maintain the python-mpv and gerbolyze python packages on PyPI.</description>
</item>
<item>
<title>Impressum</title>
<link>https://blog.jaseg.de/imprint/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://blog.jaseg.de/imprint/</guid>
<description>Impressum Sebastian Götte c/o Praxis Dr. Götte Krankenhausstr. 1a 54634 Bitburg imprint@jaseg.net Inhaltlich Verantwortlicher gem. § 55 II RStV: Sebastian Götte (Anschrift s.o.) Lizenz dieser Seite Dieses Werk ist lizenziert unter einer Creative Commons Namensnennung - Nicht-kommerziell - Weitergabe unter gleichen Bedingungen 4.0 International Lizenz (CC-BY-NC-SA) . Haftungsbeschränkung für Inhalte der Website Gemäß § 7 Abs. 1 TMG ist der Verantwortliche der Website i. S. v. § 5 TMG für eigene Informationen, die er zur Nutzung bereithält, nach den allgemeinen Gesetzen verantwortlich.</description>
</item>
</channel>
</rss>

279
posts/hsm-basics/index.html Normal file
View file

@ -0,0 +1,279 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Hardware Security Module Basics | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<div class="article-meta">
<h1><span class="title">Hardware Security Module Basics</span></h1>
<h2 class="date">2019/05/17</h2>
<p class="terms">
</p>
</div>
<main>
<div class="document">
<div class="section" id="hardware-security-modules-and-security-research-and-cryptography">
<h2>Hardware Security Modules and Security Research and Cryptography</h2>
<p>On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of
Prof. Mori's security research working group at Waseda University. The motivation for this was that outside of low-level
hardware security people and people working in the financial industry HSMs are not thought about that often. In
particular most network or systems security people would not consider them an option. Also it could turn out to be
really interesting to think about what could be done with an HSM in conjunction with modern cryptography (instead of
just plain old RSA-OAEP and AES-CBC).</p>
<p><a class="reference external" href="mori_semi_hsm_talk_web.pdf">Click here to download a PDF with the slides for this talk.</a></p>
</div>
<div class="section" id="ideas-for-research-in-hsms">
<h2>Ideas for research in HSMs</h2>
<p>Preparing for this talk brought me back to some research ideas I've been working on for a while now. Since I'm not sure
I'll find the time to properly research this topic, I thought it would be great to write down some rought outlines first
for future reference.</p>
<div class="section" id="the-problem-with-current-hsm-tech">
<h3>The Problem with current HSM tech</h3>
<p>Currently, HSMs are only used in certain specific niche applications such as certificate authority key management and
financial transaction data handling. One key reason for this is that HSMs currently don't provide the affordances that
would be needed for them to be adopted more widely by the cryptographic and security engineering community. As far as I
can tell, the two core missing affordances are:</p>
<ol class="arabic simple">
<li>To be more widely adopted, HSMs must become less expensive. Currently, they go for several tens of thousands of Euro,
which puts them outside most budgets.</li>
<li>To be more widely adopted, HSMs must provide the standardized programming interfaces familiar to cryptographic
developers. Currently, every HSM vendor has their own custom cryptographic API and a developer will have to train on
one specific vendor's tooling. Furthermore, any documentation of these internals is kept secret behind NDAs. This
constitutes a high barrier to entry, decreasing adoption in particular with young developers accustomed to
open-source ecosystems.</li>
</ol>
</div>
<div class="section" id="attacking-cost-of-implementation">
<h3>Attacking cost of implementation</h3>
<p>The first issue can be addressed by simply creating a viable low-cost alternative. There is no fundamental technical
reason for the high cost of HSMs. This cost is instead due to manufacturers trying to recoup their expenses for R&amp;D as
well as certification from the small volumes HSMs are sold in.</p>
<p>Compared to system integration and certification the pure R&amp;D cost of HSM defense mechanisms themselves is not too high
in an academic context it should be feasible to develop a sort of HSM blueprint that can then be cheaply produced by
anyone in need. Since the application areas outlined here are far from the core business areas of the clients of
established HSM vendors this would most likely not be a realistic threat to any established vendor's business and a
co-existence of both should not pose any problems in the short term.</p>
</div>
<div class="section" id="benefits-of-an-academic-hsm-standard">
<h3>Benefits of an academic HSM standard</h3>
<p>Tackling the high cost of current HSM hardware with an open-source HSM blueprint would yield
several academic advantages beyond cost reduction.</p>
<ol class="arabic simple">
<li>An open-source blueprint could serve as an academic reference design to evaluate and compare other HSM designs
against. For instance this would not only allow quantifying the effectiveness of academic security measures but also
allow an evaluation of commercial HSMs.</li>
<li>An open-source blueprint could stimulate academic research in this academically very quiet albeit commercially
important area. This research would ultimately benefit everyone employing HSMs by raising security standards in the
field. Since HSMs are never solely relied upon for overal system security both defensive and offensive security
research would yield these benefits.</li>
<li>An open-source blueprint would encourage new people to get into the field and both apply HSMs to practical problems
as well as improve HSMs themselves. Currently, this is highly discouraged due to the strictly proprietary nature of
all available systems.</li>
<li>Finally, developing an open-source HSM blueprint might yield new findings in adjacent academic areas due to the
hightly multi-disciplinary nature of security research in general and HSM design in particular.</li>
</ol>
</div>
<div class="section" id="scope-of-an-academic-hsm-standard">
<h3>Scope of an academic HSM standard</h3>
<p>An academic HSM blueprint would need to be flexible so that researchers can adapt it to their particular problem. A
modular architecture would lend itself to this flexibility. Fundamentally, there would be three components to this
architecture. First, a <strong>base</strong> containing infrastructure such as the surveillance microcontroller, power supplies,
power supply filtering and hardware DPA countermeasures, and possibly a standardized mechanical and electrical
interface.</p>
<p>Next to the base, a system integrator would put their <em>payload</em>. The nature of this payload is intentionally kept
unspecified, and it might be anything from a cryptographic microcontroller to a small embedded system such as a
raspberry pi single board computer. Keeping the <em>payload</em> open like this achieves two benefits: It gives the HSM
blueprint's user <em>their</em> familiar tooling and the hardware <em>they</em> need, allowing fast adoption. Someone well-versed in
e.g. Javascript could literally implement their cryptography in Javascript, run it on an off-the-shelf raspberry pi and
just apply the HSM blueprint around it. In addition, keeping the <em>payload</em> open reduces the scope of what needs to be
implemented. Building a general SDK on top of something like a bare ARM SoC such as a TI OMAP or a Freescale/NXP IMX
would be a considerable additional burden, on top of the actual HSM design. Keeping the <em>payload</em> open allows research
to concentrate on the actual point, the HSM design.</p>
<p>The final and most important component would be a set of <em>security measures</em> that can be combined with the base to
form the final HSM. Each of these <em>security measures</em> would entail a detailed specification of its design, manufacture
and security properties. These <em>security measures</em> could be simple things like tamper switches or potting, but could
also be complex things like security meshes.</p>
<p>Given these three components -- <em>base</em>, <em>payload</em> and <em>security measures</em> as detailed specifications any engineer should
be able to design and manufacture a HSM customized to their needs. Unifying these three components within the HSM
blueprint would be a set of reference designs. Each reference design would implement a particular parametrization of the
three architectural components with a physical hole cut out where the payload would go.. These reference designs would
for one serve to guide any implementer on the customization and integration of their own derivation from the blueprint.
In addition it would serve as an extremely simple, low-cost point of entry into the ecosystem. A curious researcher
could simply replicate the reference design and put their existing payload inside. Practically this might mean e.g. a
researcher having PCBs produced according to the design files for a reference design for a mesh-based HSM, producing
their own mesh, physically glueing a raspberry pi SBC into the middle of it, and potting the resulting system. Given the
ease of prototype PCB fabrication today this would realistically allow evaluation of HSM technologies on a budget that
is orders of magnitude less than the cost of current HSMs.</p>
</div>
</div>
<div class="section" id="research-ideas-for-tamper-detection-mechanisms">
<h2>Research ideas for tamper detection mechanisms</h2>
<p>The core component of an HSM blueprint would be a suite of tamper detection mechanisms. Following are a few ideas on how
to improve on the current state of the art of membrane tamper switches plus temperature sensors plus PCB and printed
security meshes plus potting.</p>
<div class="section" id="improvements-on-existing-techniques">
<h3>Improvements on existing techniques</h3>
<div class="section" id="light-sensors">
<h4>Light sensors</h4>
<p><strong>Advanced analog sensing</strong>
<strong>Self-test functionality</strong></p>
</div>
<div class="section" id="security-meshes">
<h4>Security meshes</h4>
<p><strong>Analog sensing</strong></p>
</div>
</div>
<div class="section" id="diy-or-small-lab-mesh-production">
<h3>DIY or small lab mesh production</h3>
<p><strong>3D metal patterning techniques</strong> refers to any technique for producing thin, patterned metal structures on a
three-dimensional plastic substrate. The basic process would consist of 3D-printing the polymer substrate, depositing a
thin metal layer on top and then patterning this metal layer. A good starting point here would be the recent work of
<a class="reference external" href="https://www.youtube.com/watch?v=Z228xymQYho">Ben Kraznow</a> on this exact thing.</p>
<p><strong>Copper filament methods</strong> would be any method embedding copper wire from a spool in some resin or other matrix. This
could mean either of a systematic approach of carefully winding or folding the copper wire into patterns or a
non-systematic approach of simply stuffing a large tangle of copper wire into a small space. The main challenge with the
former would be to find a non-tedious way of production. The main challenge with the latter would be to find process
parameters that guarantee complete coverage of the HSM without holes or other areas of lower sensitivity to intrusions.
Both approaches would require careful consideration of the overall design including the polymer resin supporting
structure to ensure sensitivity against attacks since copper wire is mechanically much stronger than the micrometre-thin
metal coatings used in patterning techniques.</p>
</div>
<div class="section" id="envelope-measurement">
<h3>Envelope measurement</h3>
<p>Finally, I think there is another set of currently under-utilized tamper-detection methods that would be very
interesting to explore. I am not aware of an academic term for these, so I am just going to dub them <em>envelope
measurement</em> here.</p>
<p>The fundamental apporach of a mesh is to build a physical security envelope (the mesh) that physically detects when it
is disturbed (open or short circuits). This approach works well but has the disadvantage that these meshes are rather
complex to manufacture since effectively every part of them is acting as a sensing element. A conceptually more complex
but in practice potentially simpler approach might be to split the functions of security envelope and sensing element.
This would mean that in place of the mesh, some form of passive element such as metal foil forms the security envelope
which is then checked for tampering using a very sensitive sensor inside. This remote-sensing approach might simplify
the manufacture of the envelope itself and thus yield a design that is more easily customized. Following are a few ideas
on how to approach this envelope measurement problem.</p>
<p><strong>Ultrasonic</strong> If the HSM is potted, a few ultrasonic transducers could be added inside the potting. With several
transducers, any one could be used to transmit ultrasound while the others measure complex phase and energy of the
signal they receive. The circuitry for this could be made fairly simple if using a static transmit frequency or a low
chirp rate by using a homodyne receiver built around a comparator fed into some timers. This approach would likely
detect any mechanical attack and would also rule out chemical attacks involving liquids (though starting from which
amount of liquid depends on receiver sensitivity). The main disadvantages might be high power consumption and cost and
size of the ultrasonic transducers. Traditional cheap transducers made for air as a transmission medium are fairly large
and might not adequately couple into potting compound. If somehow one could convince a standard small piezo element to
do the same job that would be great as far as cost and size are concerned. A concern in some fringe use cases might be
suceptibility to ambient noise, though this could easily be reduced at the expense of space and heat dissipation
capacity by adding sound dampening on the outside. A likely attack vector against this approach might be using a laser
cutter to drill a hole through the potting compound, then inserting probes carefully chosen to not couple too much
to the potting compound ultrasonically.</p>
<p><strong>Light</strong> In either an unpotted HSM or one potted with a transparent (at some wavelengths) potting compound one could
embed LEDs and photodiodes in a similar setup to the ultrasonic setup described above. In contrast to the ultrasound,
the LEDs would literally have to light up the HSM's interior and shadows might be an issue since the HSM is likely some
flat rectangular shape. A possible solution to this would be to coat both the embedded payload and the lid with some
highly reflective paint such as some glossy silver paint or simple white paint. The basic approach might be as simple as
simply turning on several LEDs distributed throughout the HSM in turn and measuring amplitude at several photodetectors,
or as complex as doing a LIDAR-like phase measurement sweeping through a range of frequencies to determine not only
absorption but also phase/distance characteristics between emitter LED and detector photodiode. Using some high-gain TIA
along with a homodyne detector (lock-in amplifier) and changing emitter intensity, very precise measurements of both
absorption and phase might be possible, as might be measurements through almost opaque, diffuse potting compounds such
as a grey epoxide resin. The main disadvantages of this method would likely be the need to thoroughly light-proof the
entire HSM (likely by wrapping it in metal foil) and the potentially high cost of transmitter and receiver circuitry
(nice TIAs aren't cheap). To be effective against attacks using e.g. very fine drills and probes the system would likely
have to be very sensitive.</p>
<p><strong>Radar</strong> Finally, one could turn to standard radar techniques to fingerprint the inside of the HSM. The goal here would
be fingerprinting instead of mapping since only changes need to be detected. In this approach one could use homodyne
detection to improve sensitivity and reduce receiver complexity, and sweep frequencies similar to an FMCW radar (but
probably without exploiting the self-demodulation effect). Besides high cost, this approach has two disadvantages.
First, such a system would likely not go beyond 24GHz or maybe 40GHz due to component availability issues. Even at 40GHz
the wavelength in the potting compound would be in the order of magnitude of several millimeters. Fine intrusions using
some tool chosen to not interact too much with the EM field inside the HSM such as a heated ceramic needle or simply a
laser cutter might not be detectable using this approach. In any case, this system would certainly not be able to detect
small holes piercing the HSM enclosure. The HSM enclosure would have to be made into an RF shield, likely by using some
metal foil in it.</p>
<p>Overall in the author's opinion these three techniques are most promising in order <em>Light</em>, <em>Ultrasonic</em>, <em>Radar</em>. Light
would prbably provide the best sensitivity at expense of some cost. Ultrasonic might be used in conjunction with light
to cover some additional angles since it is potentially very low-cost. Radar seems hard to engineer into a solution that
works reliably and also would likely be at least an order of magnitude more expensive than the other two technologies
while not providing better sensitivity.</p>
</div>
</div>
</div>
</main>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

0
content/posts/hsm-basics/mori_semi_hsm_talk_web.pdf → posts/hsm-basics/mori_semi_hsm_talk_web.pdf
View file

125
posts/index.html Normal file
View file

@ -0,0 +1,125 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Posts | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<h1>Posts</h1>
<ul>
<li>
<span class="date">2020/08/18</span>
<a href="/posts/kicad-mesh-plugin/">Kicad Mesh Plugin</a>
</li>
<li>
<span class="date">2019/06/22</span>
<a href="/posts/private-contact-discovery/">Private Contact Discovery</a>
</li>
<li>
<span class="date">2019/05/17</span>
<a href="/posts/hsm-basics/">Hardware Security Module Basics</a>
</li>
<li>
<span class="date">2018/05/19</span>
<a href="/posts/serial-protocols/">How to talk to your microcontroller over serial</a>
</li>
<li>
<span class="date">2018/05/03</span>
<a href="/posts/thors-hammer/">Thor&#39;s Hammer</a>
</li>
<li>
<span class="date">2018/05/02</span>
<a href="/posts/multichannel-led-driver/">32-Channel LED tape driver</a>
</li>
<li>
<span class="date">2018/05/02</span>
<a href="/posts/wifi-led-driver/">Wifi Led Driver</a>
</li>
<li>
<span class="date">2018/05/02</span>
<a href="/posts/led-characterization/">LED Characterization</a>
</li>
</ul>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

86
posts/index.xml Normal file
View file

@ -0,0 +1,86 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Posts on jaseg.de</title>
<link>https://blog.jaseg.de/posts/</link>
<description>Recent content in Posts on jaseg.de</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<lastBuildDate>Tue, 18 Aug 2020 13:15:39 +0200</lastBuildDate><atom:link href="https://blog.jaseg.de/posts/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Kicad Mesh Plugin</title>
<link>https://blog.jaseg.de/posts/kicad-mesh-plugin/</link>
<pubDate>Tue, 18 Aug 2020 13:15:39 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/kicad-mesh-plugin/</guid>
<description>Tamper Detection Meshes Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for groceries with a credit card, messaging a friend on Signal or unlocking a car with its keyfob would not be possible.</description>
</item>
<item>
<title>Private Contact Discovery</title>
<link>https://blog.jaseg.de/posts/private-contact-discovery/</link>
<pubDate>Sat, 22 Jun 2019 10:30:00 +0800</pubDate>
<guid>https://blog.jaseg.de/posts/private-contact-discovery/</guid>
<description>Private Contact Discovery Private Contact Discovery (PCD) is the formal name for the problem modern smartphone messenger applications have on installation: Given a user&#39;s address book, find out which of their contacts also use the same messenger without the messenger&#39;s servers learning anything about the user&#39;s address book. The widespread non-private way to do this is to simply upload the user&#39;s address book to the app&#39;s operator&#39;s servers and do an SQL JOIN keyed on the phone number field against the database of registered users.</description>
</item>
<item>
<title>Hardware Security Module Basics</title>
<link>https://blog.jaseg.de/posts/hsm-basics/</link>
<pubDate>Fri, 17 May 2019 15:29:20 +0800</pubDate>
<guid>https://blog.jaseg.de/posts/hsm-basics/</guid>
<description>Hardware Security Modules and Security Research and Cryptography On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of Prof. Mori&#39;s security research working group at Waseda University. The motivation for this was that outside of low-level hardware security people and people working in the financial industry HSMs are not thought about that often. In particular most network or systems security people would not consider them an option.</description>
</item>
<item>
<title>How to talk to your microcontroller over serial</title>
<link>https://blog.jaseg.de/posts/serial-protocols/</link>
<pubDate>Sat, 19 May 2018 08:09:46 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/serial-protocols/</guid>
<description>Scroll to the end for the TL;DR.
In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error conditions.
If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to another microcontroller or to a computer.</description>
</item>
<item>
<title>Thor&#39;s Hammer</title>
<link>https://blog.jaseg.de/posts/thors-hammer/</link>
<pubDate>Thu, 03 May 2018 11:59:37 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/thors-hammer/</guid>
<description>In case you were having an inferiority complex because your friends&#39; IBM Model M keyboards are so much louder than the shitty rubber dome freebie you got with your pc... Here&#39;s the solution: Thor&#39;s Hammer, a simple typing cadence enhancer for PS/2 keyboards.
Your browser does not support the HTML5 video tag. A demonstration of the completed project. h264 download / webm download The connects to the keyboard&#39;s PS/2 clock line and briefly actuates a large solenoid on each key press.</description>
</item>
<item>
<title>32-Channel LED tape driver</title>
<link>https://blog.jaseg.de/posts/multichannel-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:14 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/multichannel-led-driver/</guid>
<description>Theoretical basics Together, a friend and I outfitted the small staircase at Berlin&#39;s Chaos Computer Club with nice, shiny RGB-WW LED tape for ambient lighting. This tape is like regular RGB tape but with an additional warm white channel, which makes for much more natural pastels and whites. There are several variants of RGBW tape. Cheap ones have separate RGB and white LEDs, which is fine for indirect lighting but does not work for direct lighting.</description>
</item>
<item>
<title>Wifi Led Driver</title>
<link>https://blog.jaseg.de/posts/wifi-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:03 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/wifi-led-driver/</guid>
<description>Project motivation The completed driver board installed in the 3D-printed case. This device can now be connected to 12V and two segments of LED tape that can then be controlled trough Wifi. The ESP8266 module goes on the pin header on the left and was removed for this picture. After the multichannel LED driver was completed, I was just getting used to controlling LEDs at 14-bit resolution. I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases.</description>
</item>
<item>
<title>LED Characterization</title>
<link>https://blog.jaseg.de/posts/led-characterization/</link>
<pubDate>Wed, 02 May 2018 11:18:38 +0200</pubDate>
<guid>https://blog.jaseg.de/posts/led-characterization/</guid>
<description>Preface Recently, I have been working on a small driver for ambient lighting using 12V LED strips like you can get inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when fading between different colors, the colors look all wrong!</description>
</item>
</channel>
</rss>

0
content/posts/kicad-mesh-plugin/images/anim.webp → posts/kicad-mesh-plugin/images/anim.webp Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 35 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/cells-0.svg → posts/kicad-mesh-plugin/images/cells-0.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 292 KiB

After

Width:  |  Height:  |  Size: 292 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/cells-100.svg → posts/kicad-mesh-plugin/images/cells-100.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/cells-25.svg → posts/kicad-mesh-plugin/images/cells-25.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/cells-50.svg → posts/kicad-mesh-plugin/images/cells-50.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/cells-75.svg → posts/kicad-mesh-plugin/images/cells-75.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/grid-vis-plain.svg → posts/kicad-mesh-plugin/images/grid-vis-plain.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/grid-vis.svg → posts/kicad-mesh-plugin/images/grid-vis.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/kicad-mesh-outline.png → posts/kicad-mesh-plugin/images/kicad-mesh-outline.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/kicad-mesh-result-large.png → posts/kicad-mesh-plugin/images/kicad-mesh-result-large.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 193 KiB

After

Width:  |  Height:  |  Size: 193 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/kicad-mesh-settings.png → posts/kicad-mesh-plugin/images/kicad-mesh-settings.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 45 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/kicad-mesh-settings2.png → posts/kicad-mesh-plugin/images/kicad-mesh-settings2.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 128 KiB

After

Width:  |  Height:  |  Size: 128 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/maze_tiles.svg → posts/kicad-mesh-plugin/images/maze_tiles.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 189 KiB

After

Width:  |  Height:  |  Size: 189 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/maze_tiles_plain.svg → posts/kicad-mesh-plugin/images/maze_tiles_plain.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 175 KiB

After

Width:  |  Height:  |  Size: 175 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/modern_art.svg → posts/kicad-mesh-plugin/images/modern_art.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/tiles-25-small.svg → posts/kicad-mesh-plugin/images/tiles-25-small.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

Before After
Before After

0
content/posts/kicad-mesh-plugin/images/traces-25-small.svg → posts/kicad-mesh-plugin/images/traces-25-small.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 809 KiB

After

Width:  |  Height:  |  Size: 809 KiB

Before After
Before After

255
posts/kicad-mesh-plugin/index.html Normal file
View file

@ -0,0 +1,255 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Kicad Mesh Plugin | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<div class="article-meta">
<h1><span class="title">Kicad Mesh Plugin</span></h1>
<h2 class="date">2020/08/18</h2>
<p class="terms">
</p>
</div>
<main>
<div class="document">
<figure>
<img src="images/anim.webp" style="max-width: 20em">
</figure><div class="section" id="tamper-detection-meshes">
<h2>Tamper Detection Meshes</h2>
<p>Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and
mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious
surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for
groceries with a credit card, messaging a friend on <a class="reference external" href="https://signal.org">Signal</a> or unlocking a car with its keyfob
would not be possible. The security of all of these systems in its core lies on the secrecy of cryptographic keys.
Systems differ in what kind of keys they use, how often these keys are replaced and the intricacies of the cryptographic
operations these keys fit into but all have in common that their security relies on keeping the keys secret.</p>
<p>In practice, this secrecy has been implemented in many different ways. Mass-market software such as browsers or
messenger apps usually relies on some operating system facility to tell the computer &quot;<em>please keep this piece of memory
away from all other applications</em>&quot;. While on desktop operating systems usually this does not provide much of a barrier
to other programs on the same computer, on modern mobile operating systems this approach is actually quite secure.
However, given sufficient resources no security is perfect. All of these systems can be compromised if the host
operating system is compromised sufficiently, and for organizations with considerable resources a market has sprung up
that offers turn-key solutions for all wiretapping needs.</p>
<p>In some applications, this level of security has not been considered sufficient. Particularly financial infrastructure
is such a high-profile target that a lot of effort has been put into the security of cryptographic implementations. The
best cryptographic algorithm is useless if it is run on a compromised system (from that system's point of view anyway).
One of the core cryptographic components in financial applications are smartcards like they are used as payment cards in
most countries nowadays. These smartcards contain a small, specialized cryptographic microcontroller that is designed to
be hard to tamper with. Though one of the design goals of the system is to reduce the amount of sensitive information
stored on the card, things such as copying of a card can only be hindered by making the chip hard to read out.</p>
<figure>
<img src="images/modern_art.svg" style="max-width: 20em">
</figure><p>With smartcards being the means of choice on one side of the counter in electronic payments, on the other side of the
counter a different technology prevails. Attacks on payment terminals are bound to have much more dire consequences than
attacks on individual cards since one terminal might see hundreds of cards being read every day. For this reason, the
level of attack countermeasures employed in these terminals is a considerable step up from bare smartcards. While a
smartcard is made physically hard to tamper, it does not have a battery and it can only detect tampering once it is
powered by a reader. This allows for well-equipped attackers to use tools such as Focused Ion Beam (FIB) workstations to
circumvent the smartcard's defences while it is powered down, and then power up the card to carry out the actual attack.</p>
<p>The answer to this problem in electronic payment infrastructure is called <em>Hardware Security Module</em>, or HSM. An HSM is
similar to a smartcard in its function (cryptographic processing using keys that are meant to never leave the protection
of the HSM). The one major between the two is that an HSM has its own battery and is continuously powered from its
manufacture to the day it is scrapped. If the HSM looses power at any point in time, it uses a small amount of energy
stored internally to securely wipe all cryptographic secrets from its memory within a few milliseconds.</p>
<p>Being powered at all times allows the HSM to actively detect and respond to attacks. The most common way this is done is
by wrapping the juicy secret parts in a foil or a printed circuit board that is patterned with a long and convoluted
maze of wires, called a <em>mesh</em>. The HSM is continuously monitoring these wires for changes (such as shorts, breaks or
changes in resistance) and will sound the alarm when any are detected. Practically, this presents a considerable hurdle
to any attacker: They have to find a way to disable or circumvent the mesh while it is being monitored by the HSM. In
practice, often this is no insurmountable challenge but it again increases attack costs.</p>
</div>
<div class="section" id="diy-meshes">
<h2>DIY Meshes</h2>
<p>Throughout my studies in security research I have always had an interest in HSMs. I have taken apart my fair share of
HSMs and at this point, to understand the technology more, I want to experiment with building my own HSM. In last year's
<a class="reference external" href="https://blog.jaseg.de/posts/hsm-basics/">HSM basics</a> post I have lined out some ideas for a next generation design that
deviates from the bread-and-butter apporoach of using a mesh as the primary security feature. Before embarking on
practical experiments with these ideas, I want to first take a stab at replicating the current state of the art as best
I can. State of the art meshes often use exotic substrates such as 3D plastic parts with traces chemically deposited on
their surface or special flexible substrates with conductive ink traces. These technologies will likely be too
cumbersome for me to implement myself only for a few prototypes, and industrial manufacturers will most likely be too
expensive. Thus, I will concentrate on regular PCB technology for now.</p>
<p>The idea of a mesh on a PCB is pretty simple: You have one or several traces that you try to cover every corner of the
mesh PCB's area with. To be most effective, the traces should be as thin and as close together as possible. To increase
the chances of a manipulation being detected, multiple traces can also be used that can then be monitored for shorts
between them.</p>
<p>While one can feasibly lay out these traces by hand, this really is an ideal application of a simple auto-router. While
general PCB autorouting is <em>hard</em>, auto-routing just a few traces to approximate a space-filling curve is not. Since I
am just starting out, I went with the simplest algorithmic solution I could think of. I first approximate the area
designated to the mesh with a square grid whose cells are a multiple of my trace/space size. The mesh will only be drawn
into grid cells that are fully inside the set boundaries. All cells outside or going across the border are discarded in
this step.</p>
<p>I decided to implement this auto-router in a KiCAD plugin. Though KiCADs plugin API is not the best, it was just about
usable for this task.</p>
<figure>
<img src="images/kicad-mesh-outline.png" alt="KiCAD showing an irregular board shape with rounded corners and
indents. In the middle of the board there is a footprint for a 4-pin surface-mount pin header.">
<figcaption>The process starts out with the mesh shape being defined inside KiCAD. The mesh's outline is drawn
onto one of the graphical "Eco" layers. A footprint is placed to serve as a placeholder for the mesh's
connections to the outside world. This footprint is later used as the starting point for the mesh generation
algorithm.</figcaption>
</figure><figure>
<img src="images/grid-vis-plain.svg" alt="A vizualization of the grid fitting process. Over the mesh's irregular
outline a grid is drawn. In this picture, all grid cells that are fully inside the grid are shown. Grid cells
that overlap the mesh border are highlighted. Grid cells outside of the mesh border are not drawn.">
<figcaption>A visualization of the grid fitting process. First, a grid large enough to contain the mesh border
is generated. Then, every cell is checked for overlap with the mesh border area. If the cell is fully inside, it
(yellow), it is considered in the mesh generation later. Cells outside (gray) or on the border (red) are
discarded.</figcaption>
</figure><p>After generating the grid, starting from the place I want to connect to the mesh, I walk the grid's cells one by one to
generate a tree that covers the entire grid's area. To set the mesh's starting place I place a footprint on the board
(dark gray in the picture above) whose designator I then tell my script. The tree generation algorithm looks like a
depth-first search, except all checks are random. Depending on the level of randomness used at each step of the
algorithm it yields more or less organized-looking results. Below are five example runs of the algorithm at differing
levels of randomness with the cells colored according to their distance from the tree root. 0% randomness means that the
algorithm is going to try cells in forward direction first on every step, and only then try out left and right. 100%
means that on every step, the algorithm is choosing a new direction at random.</p>
<figure>
<figure class="side-by-side">
<img src="images/cells-0.svg" alt="a completely organized looking grid with spiral patterns all over.">
<figcaption>0%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-25.svg">
<figcaption>25%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-50.svg">
<figcaption>50%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-75.svg">
<figcaption>75%</figcaption>
</figure><figure class="side-by-side">
<img src="images/cells-100.svg" alt="a completely random looking grid with cells aggregating into ireggular
areas that look like paint splotches.">
<figcaption>100%</figcaption>
</figure>
</figure><p>After I have built this tree like you would do in a depth-first search, I draw my one or several mesh mesh traces into
it. The core observation here is that there is only 16 possible ways a cell can be connected: It has four neighbors,
each of which it can either be connected to or not, which results in 2^4 options. If you consider rotations and
mirroring, this works out to rotations or mirrored versions of only six base tiles: The empty tile, a tile with all four
sides connected, a straight through, a 90 degree bend, and a &quot;T&quot;-junction—see the illustration below.</p>
<figure>
<img src="images/maze_tiles_plain.svg" style="max-width: 20em">
<figcaption>
There are six possible tile types in our connectivity graph inside its square tiling. This graphic illustrates
all sixteen rotations of these with how they would look in a two-conductor mesh.
</figcaption>
</figure><p>After tiling the grid according to the key above, we get the result below.</p>
<figure>
<img src="images/tiles-25-small.svg">
<figcaption>
An auto-routed mesh with traces colored according to tile types.
</figcaption>
</figure><figure>
<img src="images/traces-25-small.svg">
<figcaption>
The same mesh, but with traces all black.
</figcaption>
</figure><p>Putting it all together got me the KiCAD plugin you can see in the screenshot below.</p>
<figure>
<img src="images/kicad-mesh-settings2.png">
<figcaption>
The plugin settings window open.
</figcaption>
</figure><figure>
<img src="images/kicad-mesh-result-large.png">
<figcaption>
After runing the plugin, the generated mesh looks like this in pcbnew.
</figcaption>
</figure><p>I am fairly happy with the result, but getting there was a medium pain. Especially KiCAD's plugin API is still very
unfinieshed. It is hard to use, most parts are completely undocumented and if you use anything but its most basic parts
things tend to break. One particular pain point for me was that after generating the mesh, the traces have been added to
the board, but are still invisible for some reason. You have to save the board first, then re-load the file for them to
become visible. Also KiCAD crashes whenever the plugin tries to remove a trace, so currently my workflow involves always
making a copy of the board file first and treating mesh generation as a non-reversible finishing step.</p>
<p><a class="reference external" href="https://git.jaseg.de/kimesh.git/tree/plugin/mesh_dialog.py">Check out the code on my cgit</a>.</p>
<!-- ::
.. raw:: html
<figure>
<img src="images/grid-vis-plain.svg" alt="">
<figcaption></figcaption>
</figure> -->
</div>
</div>
</main>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

0
content/posts/led-characterization/images/daylight_spectrum_dvd.jpg → posts/led-characterization/images/daylight_spectrum_dvd.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Before After
Before After

0
content/posts/led-characterization/images/driver_ringing_strong.jpg → posts/led-characterization/images/driver_ringing_strong.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 279 KiB

Before After
Before After

0
content/posts/led-characterization/images/driver_ringing_weak.jpg → posts/led-characterization/images/driver_ringing_weak.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 285 KiB

After

Width:  |  Height:  |  Size: 285 KiB

Before After
Before After

0
content/posts/led-characterization/images/electronics_whole.jpg → posts/led-characterization/images/electronics_whole.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 428 KiB

After

Width:  |  Height:  |  Size: 428 KiB

Before After
Before After

0
content/posts/led-characterization/images/hsv_cylinder.png → posts/led-characterization/images/hsv_cylinder.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 287 KiB

After

Width:  |  Height:  |  Size: 287 KiB

Before After
Before After

0
content/posts/led-characterization/images/photodiode_sensitivity.svg → posts/led-characterization/images/photodiode_sensitivity.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before After
Before After

0
content/posts/led-characterization/images/preamp_back.jpg → posts/led-characterization/images/preamp_back.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 332 KiB

After

Width:  |  Height:  |  Size: 332 KiB

Before After
Before After

0
content/posts/led-characterization/images/preamp_front.jpg → posts/led-characterization/images/preamp_front.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 301 KiB

After

Width:  |  Height:  |  Size: 301 KiB

Before After
Before After

0
content/posts/led-characterization/images/preamp_schematic.jpg → posts/led-characterization/images/preamp_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 271 KiB

Before After
Before After

0
content/posts/led-characterization/images/processed_plot_cheap_rgb.svg → posts/led-characterization/images/processed_plot_cheap_rgb.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Before After
Before After

0
content/posts/led-characterization/images/raw_plot_cheap_rgb.svg → posts/led-characterization/images/raw_plot_cheap_rgb.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 103 KiB

Before After
Before After

0
content/posts/led-characterization/images/rgb_cube.svg → posts/led-characterization/images/rgb_cube.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 17 KiB

Before After
Before After

0
content/posts/led-characterization/images/spectrograph_step1_parts.jpg → posts/led-characterization/images/spectrograph_step1_parts.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 278 KiB

After

Width:  |  Height:  |  Size: 278 KiB

Before After
Before After

0
content/posts/led-characterization/images/spectrograph_step2.jpg → posts/led-characterization/images/spectrograph_step2.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 271 KiB

Before After
Before After

0
content/posts/led-characterization/images/spectrograph_step3.jpg → posts/led-characterization/images/spectrograph_step3.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 250 KiB

After

Width:  |  Height:  |  Size: 250 KiB

Before After
Before After

0
content/posts/led-characterization/images/spectrograph_step4_complete.jpg → posts/led-characterization/images/spectrograph_step4_complete.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 297 KiB

After

Width:  |  Height:  |  Size: 297 KiB

Before After
Before After

0
content/posts/led-characterization/images/zeus_hammer_breadboard.jpg → posts/led-characterization/images/zeus_hammer_breadboard.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 435 KiB

After

Width:  |  Height:  |  Size: 435 KiB

Before After
Before After

0
content/posts/led-characterization/images/zeus_hammer_breadboard_original.jpg → posts/led-characterization/images/zeus_hammer_breadboard_original.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.4 MiB

After

Width:  |  Height:  |  Size: 2.4 MiB

Before After
Before After

0
content/posts/led-characterization/images/zeus_hammer_schematic.jpg → posts/led-characterization/images/zeus_hammer_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 145 KiB

After

Width:  |  Height:  |  Size: 145 KiB

Before After
Before After

0
content/posts/led-characterization/images/zeus_hammer_schematic_original.jpg → posts/led-characterization/images/zeus_hammer_schematic_original.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.8 MiB

After

Width:  |  Height:  |  Size: 1.8 MiB

Before After
Before After

469
posts/led-characterization/index.html Normal file
View file

@ -0,0 +1,469 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>LED Characterization | jaseg.de</title>
<link rel="stylesheet" href="/css/style.css" />
<link rel="stylesheet" href="/css/fonts.css" />
<header>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/atom-one-light.min.css">
<script src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/highlight.min.js"></script>
<script>hljs.initHighlightingOnLoad();</script>
<nav>
<ul>
<li class="pull-left ">
<a href="https://blog.jaseg.de/">/home/jaseg.de</a>
</li>
</ul>
</nav>
</header>
</head>
<body>
<br/>
<div class="article-meta">
<h1><span class="title">LED Characterization</span></h1>
<h2 class="date">2018/05/02</h2>
<p class="terms">
</p>
</div>
<main>
<div class="document">
<div class="section" id="preface">
<h2>Preface</h2>
<p>Recently, I have been working on a <a class="reference external" href="https://blog.jaseg.de/posts/wifi-led-driver/">small driver</a> for ambient lighting using 12V LED strips like you can get
inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up
to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when
fading between different colors, the colors look <em>all wrong</em>! This observation led me down a rabbit hole of color
perception and LED peculiarities.</p>
<p>The idea of the LED driver was that it can be used either with up to eight single-color LED tapes or, much more
interesting, with up to two RGB or RGBW (red-green-blue-white) LED tapes. For ambient lighting high color resolution was
really important so you could dim it down a lot without flickering. I ended up using the same driver stage I used in the
<a class="reference external" href="https://blog.jaseg.de/posts/multichannel-led-driver/">multichannel LED driver</a> project for its great color resolution and low hardware requirements.</p>
<figure>
<img src="images/rgb_cube.svg" alt="An illustration of the RGB color cube.">
<figcaption>An illustration of the RGB color cube.
<a href="https://commons.wikimedia.org/wiki/File:RGB_color_cube.svg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Maklaan">Maklaan from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure><p>To make setting colors over Wifi more intuitive I implemented support for HSV colors. RGB is fine for communication
between computers, but I think HSV is easier to work with when manually inputting colors from the command line. RGB is
close to how most monitors, cameras and the human visual apparatus work on a very low level but doesn't match
higher-level human color perception very well. When we describe a color we tend to think in terms of &quot;hue&quot; or
&quot;brightness&quot;, and computing a measure of those from RGB values is not easy.</p>
</div>
<div class="section" id="colors-and-color-spaces">
<h2>Colors and Color Spaces</h2>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Color_space">Color spaces</a> are a mathematical abstraction of the concept of color. When we say &quot;RGB&quot;, most of the time we actually
mean <a class="reference external" href="https://en.wikipedia.org/wiki/SRGB">sRGB</a>, a standardized notion of how to map three numbers labelled &quot;red&quot;, &quot;green&quot; and &quot;blue&quot; onto a perceived
color. <a class="reference external" href="https://en.wikipedia.org/wiki/HSL_and_HSV">HSV</a> is an early attempt to more closely align these numbers with our perception. After HSV, a number of other
<em>perceptual</em> color spaces such as <a class="reference external" href="https://en.wikipedia.org/wiki/CIE_1931_color_space">XYZ (CIE 1931)</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/Lab_color_space">CIE Lab/LCh</a> were born, further improving this alignment. In
this mathematical model, mapping a color from one color space into another color space is just a coordinate
transformation.</p>
<figure>
<img src="images/hsv_cylinder.png" alt="An illustration of the HSV color space as a cylinder.">
<figcaption>An illustration of the HSV color space as a cylinder.
<a href="https://commons.wikimedia.org/wiki/File:HSV_color_solid_cylinder.png">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:SharkD">SharkD from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure><p>CIE 1931 XYZ is much larger than any other color space, which is why it is a good basis to express other color spaces
in. In XYZ there are many coordinates that are outside of what the human eye can perceive. Below is an illustration of
the sRGB space within XYZ. The wireframe cube is (0,0,0) to (1,1,1) in XYZ. The colorful object in the middle is what
of sRGB fits inside XYZ, and the lines extending out from it indicate the space that can be expressed in sRGB but not in
XYZ. The fat white curve is a projection of the <em>monochromatic spectral locus</em>, that is the curve of points you get in
XYZ for pure visible wavelengths.</p>
<p>As you can see, sRGB is <em>much</em> smaller than XYZ or even the part within the monochromatic locus that we can perceive. In
particular in the blues and greens we loose <em>a lot</em> of colors to sRGB.</p>
<figure>
<video controls loop>
<source src="video/sRGB.mkv" type="video/h264">
<source src="video/sRGB.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured sRGB color space within XYZ. The thick, white line is the spectral
locus.
<a href="video/sRGB.mkv">mkv/h264 download</a> /
<a href="video/sRGB.webm">webm download</a>
</figcaption>
</figure><p>The wrong colors I got when fading between colors were caused by this coordinate transformation being askew. Thinking
over the problem, there are several sources for imperfections:</p>
<ul class="simple">
<li>The LED driver may not be entirely linear. For most modulations such as PWM the brightness will be linear starting
from a certain value, but there is probably an offset caused by imperfect edges of the LED current. This offset can be
compensated with software calibration. I built a calibration setup for driver linearity in the <a class="reference external" href="https://blog.jaseg.de/posts/multichannel-led-driver/">multichannel LED
driver</a> project. Below are pictures of ringing on the edges of an LED driver's waveform.</li>
<li>The red, green and blue channels of the LEDs used on the LED tape are not matched. This skews the RGB color space.
In practice, the blue channel of my RGB tape to me <em>looks</em> much brighter than the red channel.</li>
<li>The precise colors of the red, green and blue channels of the LEDs are unknown. Though the red channel <em>looks</em> red, it
may be of a slightly different hue compared to the reference red used in <a class="reference external" href="https://en.wikipedia.org/wiki/SRGB">sRGB</a> which would also skew the RGB color
space.</li>
</ul>
<figure>
<figure class="side-by-side">
<img src="images/driver_ringing_strong.jpg" alt="Strong ringing on the LED voltage waveform edge at about
100% overshoot during about 70% of the cycle time.">
<figcaption>The LED strip being at the end of a couple meters of wire caused extremely bad ringing at high
driving frequencies.</figcaption>
</figure><figure class="side-by-side">
<img src="images/driver_ringing_weak.jpg" alt="Weak ringing on the LED voltage waveform edge at about 30%
overshoot during about 20% of the cycle time.">
<figcaption>Adding a resistor in front of the MOSFET gate to slow the transition dampened the ringing
somewhat, but ultimately it cannot be eliminated entirely.</figcaption>
</figure>
</figure><p>These last two errors are tricky to compensate. What I needed for that was basically a model of the <em>perceived</em> colors
of the LED tape's color channels. A way of doing his is to record the spectra of all color channels and then evaluate
their respective XYZ coordinates. If all three channels are measured in one go with the same setup the relative
magnitudes of the channels in XYZ will be accurate.</p>
<p>To map any color to the LEDs, the color's XYZ coordinates simply have to be mapped onto the linear coordinate system
produced by these three points within XYZ. LEDs are mostly linear in their luminous flux vs. current characteristic so
this model will be adequate. The spectral integrals mapping the channels' measured responses to XYZ need only be
calculated once and their results can be used as scaling factors thereafter.</p>
</div>
<div class="section" id="measuring-the-spectrum">
<h2>Measuring the spectrum</h2>
<p>In order to compensate for the cheap LED tape's non-ideal performance I had to measure the LED's red, green and blue
channels' spectra. The obvious thing would be to go out and buy a <a class="reference external" href="https://en.wikipedia.org/wiki/Ultraviolet%E2%80%93visible_spectroscopy">spectrograph</a>, or ask someone to borrow theirs. The
former is kind of expensive, and I did not want to wait two weeks for the thing to arrive. The latter I could probably
not do every time I got new LED tape. Thus the only choice was to build my own.</p>
<p>Luckily, building your own spectrometer is really easy. The first thing you need is something that splits incident light
into its constituent wavelengths. In professional devices this is called the <em>`monochromator`_</em>, since it allows extraction
of small color bands from the spectrum. The second thing is some sort of optics that project the incident light onto a
screen behind the monochromator. In professional devices lenses or curved mirrors are used. In a simple homebrew job a
pinhole as you would use in a <a class="reference external" href="https://en.wikipedia.org/wiki/Pinhole_camera">camera obscura</a> does a remarkably nice job.</p>
<p>For the monochromator component several things could be used. A prism would work, but I did not have any. The
alternative is a <a class="reference external" href="https://en.wikipedia.org/wiki/Diffraction_grating">diffraction grating</a>. Professional gratings are quite specialized pieces of equipment and thus
rather expensive. Luckily, there is a common household item that works almost as well: A regular CD or DVD. The
microscopic grooves that are used to record data in a CD or DVD work the same as the grooves in a professional
diffraction grating.</p>
</div>
<div class="section" id="household-spectra">
<h2>Household spectra</h2>
<p>From this starting point, a few seconds on my favorite search engine yielded an <a class="reference external" href="http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf">article by two researchers from the
National Science Museum in Tokyo</a> providing a nice blueprint for a simple cardboard-and-DVD construction for use in
classrooms. I replicated their device using a DVD and it worked beautifully. Daylight and several types of small LEDs I
had around did show the expected spectra. Small red, yellow, green, and blue LEDs showed narrow spectra, daylight one
continuous broad one, and white LEDs a continuous broad one with a distinct bright spot in the blue part. The
single-color LED spectra are quite narrow since they are determined by the LED's semiconductor's band gap, which is
specific to the semiconductor used and is quite precise. White LEDs are in fact a blue LED chip covered with a so-called
<em>phosphor</em>. This phosphor is not elementary phosphorus but an anorganic compound that absorbs the LED chip's blue light
and re-emits a broader spectrum of more yellow-ish wavelengths instead. The final LED spectrum is a superposition of
both spectra, with some of the original blue light leaking through the phosphor mixing with the broadband yellow
spectrum of the phosphor.</p>
<figure>
<figure class="side-by-side">
<img src="images/spectrograph_step1_parts.jpg">
<figcaption>The ingredients. The cup of coffee and Madoka Magica DVD set are essential to the eventual
function of the appartus.</figcaption>
</figure><figure class="side-by-side">
<img src="images/spectrograph_step2.jpg">
<figcaption>Step 1: Cut to size and mark down all holes as described in <a
href="http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf">the manual</a></figcaption>
</figure>
<figure class="side-by-side">
<img src="images/spectrograph_step3.jpg">
<figcaption>Step 2: Cut out all holes</figcaption>
</figure><figure class="side-by-side">
<img src="images/spectrograph_step4_complete.jpg">
<figcaption>The finished result with the back side showing. The viewing window is on the bottom of the other
side.</figcaption>
</figure>
</figure><p>Now that I had a spectrograph, I needed a somewhat predictable way of measuring the spectrum it gave me.</p>
</div>
<div class="section" id="measuring-a-spectrum">
<h2>Measuring a spectrum</h2>
<p>Pointing a camera at the spectrograph would be the obvious thing to do. This produces pretty images but has one critical
flaw: I wanted to acquire quantitative measurements of brightness across the spectrum. Since I don't have a precise
technical datasheet specifying the spectral response of any of my cameras I can't compare the absolute brightness of
different colors on their pictures. Some other sensor was needed.</p>
<figure>
<img src="images/daylight_spectrum_dvd.jpg">
<figcaption>The daylight spectrum as seen using a DVD as a grating.
<a href="https://commons.wikimedia.org/wiki/File:SpectresSolaires-DVD.jpg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Xofc">Xofc from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA 4.0</a>
</figcaption>
</figure><div class="section" id="measuring-light-intensity">
<h3>Measuring light intensity</h3>
<p>Looking around my lab, I found a bag of <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> visible-light photodiodes. Their
datasheet includes their spectral response so I can compensate for that, allowing precise-ish absolute intensity
measurements. Just like LEDs, photodiodes are extremely linear across several orders of magnitude. The datasheet of the
classic <a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> photodiode shows that this photodiode's light current is exactly proportional to illuminance over at
least three orders of magnitude. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> datasheet does not include a similar graph but its performance will be
similar. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> photodiodes I had at hand were perfect for the job compared to the vintage <a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> since their
active sensing area is really small (0.6mm by 0.6mm) compared to the BPW34 (a whopping 3mm by 3mm). If I were to use a
<a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> I would have to insert some small apterture in front of it so it does not catch too broad a part of the
spectrum at once. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> is small enough that if I just point it at the projected spectrum directly I will
already get only a small part of the spectrum inside its 0.6mm active area.</p>
<p>To convert the photodiode's tiny photocurrent into a measurable voltage I built another copy of the <a class="reference external" href="https://en.wikipedia.org/wiki/Transimpedance_amplifier">transimpedance
amplifier</a> circuit I already used in the <a class="reference external" href="https://blog.jaseg.de/posts/multichannel-led-driver/">multichannel LED driver</a>. A <a class="reference external" href="https://en.wikipedia.org/wiki/Transimpedance_amplifier">transimpedance amplifier</a> is an
amplifiert that produces a large voltage from a small current. The weird name comes from the fact that it works kind of
like an amplified resistor (which can be generalized as an <em>impedance</em> electrically). Apply a current to a resistor and
you get a voltage. A transimpedance amplifiert does the same with the difference that its input always stays at 0V,
making it look like an ideal current sink to the connected current source.</p>
<p>Transimpedance amplifiers are common in optoelectronics to convert small photocurrents to voltages. In this instance I
built a very simple circuit with a dampened transimpedance amplifier stage followed by a simple RC filter for noise
rejection and a regular non-inverting amplifier using another op-amp from the same chip to further boost the filtered
transimpedance amplifier output. I put all the passives setting amplifier response (the gain-setting resistors and the
filter resistor and capacitors) on a small removable adapter so I could easily change them if necessary. I put a small
trimpot on the virtual ground both amplifers use as a reference so I could trim that if necessary.</p>
<figure>
<img src="images/preamp_schematic.jpg" alt="A drawing of the photodiode preamplifier's schematic">
<figcaption>The photodiode preamplifier schematic. Schematic drawn with an unlicensed copy of
DaveCAD.</figcaption>
</figure><p>Following are pictures of the preamplifier board. The connectors on the top-left side are two copies of the analog
signal for the ADC and a small panel meter. The SMA connector is used as the photodiode input since coax cables are
generally low-leakage and have built-in shielding. The circuit is powered via the micro-USB connector and the analog
ground bias voltage can be adjusted using the trimpot.</p>
<p>For easy replacement, all passives setting gain and frequency response are on a small, pluggable carrier PCB made from a
SMD-to-DIP adapter.</p>
<p>Flying-wire construction is just fine for this low-frequency circuit. In a high-speed photodiode preamp, the
transimpedance amplifier circuit would be highly sensitive to stray capacitance, but we're not aiming at high speed
here.</p>
<figure>
<figure class="side-by-side">
<img src="images/preamp_front.jpg">
<figcaption>The front side of the preamplifier board.</figcaption>
</figure><figure class="side-by-side">
<img src="images/preamp_back.jpg">
<figcaption>The wiring of the photodiode preamp.</figcaption>
</figure>
</figure><p>Given a way to measure intensity what remains missing is a way to scan a single photodiode across the spectrum.</p>
</div>
<div class="section" id="scanning-the-projection">
<h3>Scanning the projection</h3>
<p>A cheap linear stage can be found in any old CD or DVD drive. These drives use a small linear stage based on a
stepper-driven screw to move the laser unit radially. Removing the laser unit and connecting a leftover stepper driver
module I was left with a small linear stage with about 45 steps per cm without microstepping enabled. The driver I used
was an <a class="reference external" href="https://www.pololu.com/file/0J450/A4988.pdf">A4988</a> module that required at least 8V motor drive voltage. I used a small micro USB-input boost converter
module to generate a stable 10V supply for the motor driver, with the USB's 5V rail used as a logic supply for the motor
driver.</p>
<p>The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> can easily be mounted to the linear stage using a small SMD breakout board glued in place with thin wires
connecting it to the transimpedance amplifier. The DVD drive linear stage is not very strong so it is important that
this wire does not put too much strain on it.</p>
<p>Above the photodiode, I mounted a small piece of paper on the linear stage to be used as a projection screen to align
the linear stage in front of the spectrometer viewing window. A line on the screen paper points to the photodiode die in
parallel to the linear stage allowing precise alignment.</p>
<p>The whole unit with photodiode preamplifier, linear stage, photodiode and stepper motor driver finally looks like this:</p>
<figure>
<img src="images/electronics_whole.jpg" alt="The complete electronics setup of the spectrograph. In the back
there is the DVD drive stepper stage. In front of it, mounted on a piece of wood are a small USB-to-12V
switching-regulator module to power the stepper motor in the top left, below on the bottom left is the
photodiode preamp and on the right is a breadboard with the stepper driver module and lots of jumper wires
interconnecting everything. On the right of the breadboard, a buspirate is attached to interface everything to a
computer. On the bottom edge of the piece of wood, two LED panel meters are mounted for readout of the preamp
output and the stepper supply voltages.">
<figcaption>The complete electronics setup. The buspirate on the right interfaces to a computer and controls the
stepper driver and ADC'es the preamp output. The two panel meters show the preamp output and stepper voltage for
setup.</figcaption>
</figure><p>The projection of the spectrum can be adjusted by moving the light source relative to the entry slot and by moving
around the grating DVD.</p>
</div>
<div class="section" id="the-capture-process">
<h3>The capture process</h3>
<p>To capture a spectrum, first the light source has to be mounted near the spectrograph's entry slot. The LED tape I
tested I just taped face-down directly into it. Next, the grating DVD has to be adjusted to make sure the spectrum
covers a sensible part of the photodiode's path. Mostly, this boils down to adjusting the photodiode distance and height
to match the vertical extent and wiggling the grating DVD to adjust the projection's horizontal position.</p>
<p>After the optics are set-up, the photodiode preamplifier has to be adjusted. In my experiments, most LED tape at 5GΩ
required a high-ish amplification. The goal in this step is to maximize the peak response of the preamp to be just
shy of its VCC rail to make best use of its dynamic range. To adjust the pre-amp, I took several very coarsely-spaced
measurements to give me an estimate of the peak while I did not yet know its precise location.</p>
<p>Since stray daylight totally swamped out the weak projection of the LED's spectrum I shielded the entire setup with a
small box made of black cardboard and two black t-shirts on top. This shielding proved adequate for all my measurements
but I had to be careful not to accidentially move the DVD that was stuck into the spectrograph with the shielding
t-shirts.</p>
<p>For capturing a single spectrum I wrote a small python script that will automatically move the stepper in adjustable
intervals and take two measurements at each point, one with the LED tape off that can be used for offset calibration and
one with the LED tape on. All measurements are stored in a sqlite database that can then be accesssed from other
scripts.</p>
<p>I built a small script that shows the progress of the current run and an jupyter notebook for data analysis. The jupyter
notebook is capable of live-updating a graph with the in-progress spectrum's data. This was quite useful as a sanity
check for when I made some mistake easy to spot in the resulting data.</p>
<p>After one color channel is captured, the LED tape has to be manually set to the next color and the next measurement can
begin.</p>
<figure>
<img src="images/raw_plot_cheap_rgb.svg" alt="A plot with three wide peaks, two large peaks on both sides and
one smaller one in the middle. The middle one overlaps the two on the sides. The large ones are about 2.5V in
amplitude. Overall, the plot is about 300 stepper steps wide with each peak being around 130 steps wide.">
<figcaption>A plot of the raw preamp output voltage versus stepper position. From left to right, the three peaks
are blue, green and red. Step 0 corresponds to the bottommost stepper position and the shortest wavelength.
</figcaption>
</figure></div>
<div class="section" id="data-analysis">
<h3>Data analysis</h3>
<p>Data analysis consists of three major steps: Offset- and stray light removal, wavelength and amplitude calibration and
color space mapping.</p>
<div class="section" id="offset-removal">
<h4>Offset removal</h4>
<p>The first task is to remove the offset caused by dark current as well as stray light of the LED's bright primary
reflection on the DVD. The LED is very bright and only a small part of its light gets reflected by the grating towards
the photodiode screen. The remaining part of the light is reflected onto the table in front of the DVD spectrograph.
Though I covered all of this with black cardboard, some of that light ultimately gets reflected onto the photodiode.
This causes a large offset, in particular in the blue part of the spectrum since in this part the photodiode is closest
to the spectrograph's opening.</p>
<p>The composite offset can be approximated with a second-order polynomial that is fitted to all the data outside of the
main peak's area. Since at this point the wavelength of each data point is still unknown this is done with a rough first
estimate of the three colors' peaks' locations and widths.</p>
</div>
<div class="section" id="wavelength-and-amplitude-calibration">
<h4>Wavelength- and amplitude calibration</h4>
<p>The photodiode's response is strongly wavelength-dependent. In particular in the blue band, the photodiode's sensitivity
gets very poor down to about 20% at the edge to ultraviolet. This effect is strong enough to move the apparent location
of the blue peak towards red.</p>
<figure>
<img src="images/photodiode_sensitivity.svg" alt="A plot of photodiode sensitivity against wavelength relative
to peak sensitivity at 820nm. The sensitivity rises from 20% at 380nm approximately linearly to 80% at 620nm,
then the rise rolls off.">
<figcaption>A plot of the photodiode's relative sensitivity in the visible spectrum. The sensitivity is
normalized against its peak at 820nm.
</figcaption>
</figure><p>The problem is that in order to remove this non-linearity, we would already have to know the wavelength of the measured
light. Since I don't, I settled for a two-step process. First, a coarse wavelength calibration is done relative to the
red peak and the short-wavelength edge of the blue peak. The photodiode measurements are then sensitivity-corrected
using this coarse measurement. Then all three channel peaks are measured in the resulting data and a fine wavelength
estimate is produced by a least-squares fit of a linear function. This fine estimate is then used for a second
sensitivity correction of all original measurements and the scale is changed from stepper motor step count to
wavelength in nanometers.</p>
<figure>
<img src="images/processed_plot_cheap_rgb.svg" alt="A plot with three wide peaks, all three of different
heights. The leftmost peak is highest at 6nA, the middle peak lowest at 1.6nA and the rightmost peak in between
at 4nA. The middle one overlaps the two on the sides. Overall, the plot spans about 300nm on its x axis with
each peak being around 100nm wide.">
<figcaption>A plot of the processed measurements. From left to right, the three peaks are blue, green and red.
</figcaption>
</figure><!-- FIXME re-do these measurements, avoiding clipping -->
<!-- FIXME re-do calibration using CCFL -->
<!-- FIXME calibration for brightness imbalance due to wedge-shaped projection of spectrum -->
</div>
<div class="section" id="color-space-mapping">
<h4>Color space mapping</h4>
<p>Finally, to achieve the objective of measuring the LED tape's channels' precise color coordinates the measured spetra
have to be matched against the color spaces' <em>color matching functions</em>. The color matching functions describe how
strong the color space's idealized <em>standard observer</em> would react to light at a particular wavelength. Going from a
measured spectrum to color coordinates XYZ works by integrating over the product of the measurement and each color
coordinate's color matching function.</p>
<p>The result are three color coordinates X, Y and Z for each channel R, G and B yielding nine coordinates in total. When
written as a matrix conversion between XYZ color space and LED-RGB color space is as simple as multiplying that matrix
(or its inverse) and a vector from one of the color spaces.</p>
<p>In XYZ space, the set of colors that can be produced with this LED tape is described by the <a class="reference external" href="https://en.wikipedia.org/wiki/Parallelepiped">parallelepiped</a> spanned by
the three channel's XYZ vectors. In the following figures, you can see a three-dimensional model of the RGB LED's color
space (colorful) as well as sRGB (white) for comparison plotted within CIE 1931 XYZ. There is no natural map to scale
both so for this illustration the LED color space has been scaled to fit. These figures were made with blender and a few
lines of python. The blender project file including all settings and the python script to generate the color space
models can be found in the <a class="reference external" href="https://github.com/jaseg/led_drv">project repo</a>.</p>
<figure>
<video controls loop>
<source src="video/led_within_srgb_scale=1.0.mkv" type="video/h264">
<source src="video/led_within_srgb_scale=1.0.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space scaled to fit within XYZ with sRGB (light gray) for
comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_scale=1.0.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_scale=1.0.webm">webm download</a>
</figcaption>
</figure><p>As you can see, the result is pretty disappointing. The LED's color space parallepiped is very narrow, which is because
the blue channel is much brighter than the other two channels. An easy fix for this is to scale-up the RGB space and
drop any values outside XYZ. The scaling factor is a trade-off between color space coverage and brightness. You can
produce the most colors when you clip all channels to brightness of the weakest channel (green in this case), but that
will make the result very dim. Scaling brightness like that stretches the RGB parallelepiped along its major axis. Up to
a point the number of possible colors (the gamut) increases at expense of maximum brightness. When the parallelepiped is
stretched far enought for all three channel vectors to be outside the 1,1,1 XYZ-cube, maximum brightness continues to
decrease but the gamut stays constant. I don't know a simple scientific way to solve this problem, so I just played
around with a couple of factors and settled on 2.5 as a reasonable compromise. Below is an illustration.</p>
<figure>
<video controls loop>
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv" type="video/h264">
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space at scale factor 2.5 within XYZ with sRGB (light gray)
for comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.webm">webm download</a>
</figcaption>
</figure></div>
</div>
</div>
<div class="section" id="firmware-implementation">
<h2>Firmware implementation</h2>
<p>In the end, the above measurements yield two matrices: One for mapping XYZ to RGB, and one for mapping RGB to XYZ. Of
the several versions of CIE XYZ I chose the CIE 1931 XYZ color space as a basis for the firmware because it is most
popular. Mapping a color coordinate in one color space to the other is as simple as performing nine floating-point
multiplications and six additions. Mapping Lab or Lch to RGB is done by first mapping Lab/Lch to XYZ, then XYZ to RGB.
Lab to XYZ is somewhat complex since it requires a floating-point power for gamma correction, but any self-respecting
libc will have one of those so this is still no problem. Lch also requires floating-point sine and cosine functions, but
these should still be no problem on most hardware.</p>
<p>My implementation of these conversions in the ESP8266 firmware of my <a class="reference external" href="https://blog.jaseg.de/posts/wifi-led-driver/">Wifi LED driver</a> can be found <a class="reference external" href="https://github.com/jaseg/esp_led_drv/blob/master/user/led_controller.c">on Github</a>. You
can view the Jupyter notebook most of the analysis above <a class="reference external" href="http://nbviewer.jupyter.org/github/jaseg/led_drv/blob/master/doc/Spectrum%20Measurement.ipynb">here</a>.</p>
</div>
</div>
</main>
<footer>
<script>
(function() {
function center_el(tagName) {
var tags = document.getElementsByTagName(tagName), i, tag;
for (i = 0; i < tags.length; i++) {
tag = tags[i];
var parent = tag.parentElement;
if (parent.childNodes.length === 1) {
if (parent.nodeName === 'A') {
parent = parent.parentElement;
if (parent.childNodes.length != 1) continue;
}
if (parent.nodeName === 'P') parent.style.textAlign = 'center';
}
}
}
var tagNames = ['img', 'embed', 'object'];
for (var i = 0; i < tagNames.length; i++) {
center_el(tagNames[i]);
}
})();
</script>
<div id="license-info">
&#169;2020 by Jan Götte. This work is licensed under
<a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC-BY-NC-SA 4.0</a>.
</div>
<div id="imprint-info">
<a href="/imprint">Impressum und Haftungsausschluss und Datenschutzerklärung</a>.
</div>
</footer>
</body>
</html>

0
content/posts/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.mkv → posts/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.mkv
View file

0
content/posts/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.webm → posts/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.webm
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=1.0.mkv → posts/led-characterization/video/led_within_srgb_scale=1.0.mkv
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=1.0.webm → posts/led-characterization/video/led_within_srgb_scale=1.0.webm
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=2.5.mkv → posts/led-characterization/video/led_within_srgb_scale=2.5.mkv
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=2.5.webm → posts/led-characterization/video/led_within_srgb_scale=2.5.webm
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=3.mkv → posts/led-characterization/video/led_within_srgb_scale=3.mkv
View file

0
content/posts/led-characterization/video/led_within_srgb_scale=3.webm → posts/led-characterization/video/led_within_srgb_scale=3.webm
View file

0
content/posts/led-characterization/video/sRGB.mkv → posts/led-characterization/video/sRGB.mkv
View file

0
content/posts/led-characterization/video/sRGB.webm → posts/led-characterization/video/sRGB.webm
View file

0
content/posts/led-characterization/video/scale=1.mkv → posts/led-characterization/video/scale=1.mkv
View file

0
content/posts/led-characterization/video/scale=1.webm → posts/led-characterization/video/scale=1.webm
View file

0
content/posts/led-characterization/video/scale=2.5.mkv → posts/led-characterization/video/scale=2.5.mkv
View file

0
content/posts/led-characterization/video/scale=2.5.webm → posts/led-characterization/video/scale=2.5.webm
View file

0
content/posts/led-characterization/video/scale=5.mkv → posts/led-characterization/video/scale=5.mkv
View file

0
content/posts/led-characterization/video/scale=5.webm → posts/led-characterization/video/scale=5.webm
View file

0
content/posts/multichannel-led-driver/images/asymmetric_iled.svg → posts/multichannel-led-driver/images/asymmetric_iled.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 69 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/asymmetric_vgate.svg → posts/multichannel-led-driver/images/asymmetric_vgate.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/bcm_schema.jpg → posts/multichannel-led-driver/images/bcm_schema.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.6 MiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/corrected_brightness_sim.svg → posts/multichannel-led-driver/images/corrected_brightness_sim.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/driver_linearity_raw.svg → posts/multichannel-led-driver/images/driver_linearity_raw.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/driver_output_ltspice_schematic.jpg → posts/multichannel-led-driver/images/driver_output_ltspice_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 856 KiB

After

Width:  |  Height:  |  Size: 856 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/driver_pcb_built.jpg → posts/multichannel-led-driver/images/driver_pcb_built.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.2 MiB

After

Width:  |  Height:  |  Size: 1.2 MiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/driver_ringing_strong.jpg → posts/multichannel-led-driver/images/driver_ringing_strong.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 279 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/driver_ringing_weak.jpg → posts/multichannel-led-driver/images/driver_ringing_weak.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 285 KiB

After

Width:  |  Height:  |  Size: 285 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/led_strip_alight.jpg → posts/multichannel-led-driver/images/led_strip_alight.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/linearization_setup.jpg → posts/multichannel-led-driver/images/linearization_setup.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.9 MiB

After

Width:  |  Height:  |  Size: 1.9 MiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/olsndot_output_schematic.jpg → posts/multichannel-led-driver/images/olsndot_output_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/olsndot_pcb.png → posts/multichannel-led-driver/images/olsndot_pcb.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 134 KiB

After

Width:  |  Height:  |  Size: 134 KiB

Before After
Before After

0
content/posts/multichannel-led-driver/images/olsndot_schematic.png → posts/multichannel-led-driver/images/olsndot_schematic.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 354 KiB

After

Width:  |  Height:  |  Size: 354 KiB

Before After
Before After

Some files were not shown because too many files have changed in this diff Show more