Jørgensen.org.uk/2018-09-26T00:50:55+01:00Homework for a 9-year-old2018-09-26T00:40:25+01:002018-09-26T00:50:55+01:00Karl E. J&oslash;rgensentag:None,2018-09-26:/homework-for-a-9-year-old.html<p>A <a href="https://www.facebook.com/groups/stanfordlehope/permalink/1869988459703153/">post about some maths homework</a> caused some consternation and confusion on one of my local facebook groups:</p> <p><img alt="homework" src="/Thoughts/homework-small.jpg"></p> <p>Lots of people seemed to think this was far too advanced for a 9-year-old, or that solving such puzzles would have no applications in real life.</p> <p>I beg to disagree.</p> <p>Yes, I …</p><p>A <a href="https://www.facebook.com/groups/stanfordlehope/permalink/1869988459703153/">post about some maths homework</a> caused some consternation and confusion on one of my local facebook groups:</p> <p><img alt="homework" src="/Thoughts/homework-small.jpg"></p> <p>Lots of people seemed to think this was far too advanced for a 9-year-old, or that solving such puzzles would have no applications in real life.</p> <p>I beg to disagree.</p> <p>Yes, I admit that solving the puzzles takes a sizeable number of steps, and might test the concentration of your little ones, but the steps themselves are easy to understand and easy to execute. They <em>do</em> require a decent understanding of numbers though.</p> <p>The underlying principle of solving <em>any</em> maths equation is to use the known facts to derive other things which we can treat as new facts. And then using <em>those</em> facts to derive even more stuff.</p> <p>In other words: Common sense. You can formalise this into <a href="https://en.wikipedia.org/wiki/Propositional_calculus">propositional calculus</a>, <a href="https://en.wikipedia.org/wiki/First-order_logic">first-order logic</a> or whatever you like, but at the end of the day <em>it is common sense</em>. Or rather: common sense on stereoids.</p> <p>In this sense, the puzzles presented to the 9-year-olds are less about formal maths than general problem solving: Using what you already know to figure out stuff. Which then allows you to figure out <em>more</em> stuff. Rinse and repeat ad nauseum. Surely this is something an education system is meant to do!</p> <p>It was presented on facebook without much context, but there are some reasonable assumptions to make here:</p> <ul> <li> <p>each letter represents a single digit</p> </li> <li> <p>two-digit numbers are represented as two letters. This is contrary to the normal maths notation where there would be an implicit multiplication</p> </li> <li> <p>In each table, each letter stands for a <em>different</em> digit. No two letters stand for the same digit within the same table.</p> </li> <li> <p>With only 9 distinct letters in each table (except for Table Y), some digit must be left out - we do not know which one, but zero sounds like a reasonable guess. Nobody likes zeros anyway. They don't matter.</p> </li> <li> <p>The goal of the puzzle is to figure out which letters represents which digits</p> </li> </ul> <p>A quick refresher on notation:</p> <ul> <li> <p>The <span class="math">$$\Rightarrow$$</span> symbol should be read as "implies that". So <span class="math">$$A \Rightarrow B$$</span> can be read as "A implies B": If A is true, then B is also true. And if A is false, then B is also false.</p> </li> <li> <p><span class="math">$$\times$$</span> is the multiplication symbol. But you did know that, right?</p> </li> </ul> <h1>Table W</h1> <p>This is the easiest one to solve - it's basically a taster:</p> <p><img alt="Table W" src="/Thoughts/table-w.jpg"></p> <p>With the assumptions above, we can focus in on <span class="math">$$H \times H = H$$</span> : this means <span class="math">$$H$$</span> can only be zero or one, no other numbers are possible. And since e.g. <span class="math">$$H \times A = A$$</span>, this means that <span class="math">$$H$$</span> cannot be zero. Hence <span class="math">$$H = 1$$</span>.</p> <p>And that's it! The remaining letters can represent whatever digits you like (including zero), and the equations will all be true.</p> <h1>Table X</h1> <p>Here we step up the difficulty level slightly:</p> <p><img alt="Table X" src="/Thoughts/table-x.jpg"></p> <p>Yes: the authors skipped the letter <em>E</em> for some reason....</p> <p>From <span class="math">$$D \times A = D$$</span> we can conclude that <span class="math">$$A = 1$$</span>.</p> <p>And considering <span class="math">$$D \times D = JA \Rightarrow D \times D = J1$$</span> : So <span class="math">$$D$$</span> times itself must result in a two-digit number whose last digit is <span class="math">$$1$$</span> : The only possiblity here is <span class="math">$$D=9, J=8$$</span></p> <p>Plugging in those values into the remaining equations revals the value of some more letters:</p> <p><span class="math">$$D \times I = AJ \Rightarrow 9 \times I = 18 \Rightarrow I = 2$$</span></p> <p><span class="math">$$D \times J = HI \Rightarrow 9 \times 8 = H2 \Rightarrow H = 7$$</span></p> <p>We can then use the newly found value of <span class="math">$$I$$</span> and <span class="math">$$H$$</span> in one of the other equations:</p> <p><span class="math">$$D \times C = IH \Rightarrow 9 \times C = 27 \Rightarrow C = 3$$</span></p> <p><span class="math">$$D \times H = FC \Rightarrow 9 \times 7 = F3 \Rightarrow F = 6$$</span></p> <p><span class="math">$$D \times F = BG \Rightarrow 9 \times 6 = BG \Rightarrow B=5, G=4$$</span></p> <p>And we then know the values of all the letters - even though we haven't "used" all the equations yet.</p> <h1>Table Y</h1> <p>Here we step up the difficulty level even futher. We have 10 letters, but still only 9 equations:</p> <p><img alt="Table Y" src="/Thoughts/table-y.jpg"></p> <p>From <span class="math">$$C \times B = C$$</span> we can conclude that <span class="math">$$B = 1$$</span>. Yes, the devious teachers <em>hid it</em> in the middle, rather than presenting it as the first equation! They expect you to understand the principles by now!</p> <p>But then it gets slightly trickier...</p> <p>Consider <span class="math">$$C \times C = AC$$</span> : The square of <span class="math">$$C$$</span> is a two-digit number with <span class="math">$$C$$</span> as its last digit. Hence <span class="math">$$C = 5$$</span> or <span class="math">$$C = 6$$</span>, but we do not know which one yet.</p> <p>If we were to guess at <span class="math">$$C = 5$$</span>, then the last digit of any of the products must be either <span class="math">$$0$$</span> or <span class="math">$$5$$</span> - which is not the case (we would only see two possible values as the right-most digit on the right-hand side of the equations, but there are obviously more than that). Hence, <span class="math">$$C$$</span> cannot be <span class="math">$$5$$</span> after all. So it must be the case that <span class="math">$$C = 6$$</span>.</p> <p>If <span class="math">$$C = 6$$</span> and <span class="math">$$C \times C = AC$$</span> then <span class="math">$$A = 3$$</span>.</p> <p>As before, we can take the values we know so far and plug into one of the remaining equations to develop more knowledge:</p> <p><span class="math">$$C \times A = BE \Rightarrow 6 \times 3 = BE \Rightarrow 6 \times 3 = 18 \Rightarrow B=1, E=8$$</span></p> <p><span class="math">$$C \times E = DE \Rightarrow 6 \times 8 = D8 \Rightarrow D = 4$$</span></p> <p><span class="math">$$C \times D = GD \Rightarrow 6 \times 4 = G4 \Rightarrow G = 2$$</span></p> <p><span class="math">$$C \times I = DG \Rightarrow 6 \times I = 42 \Rightarrow I = 7$$</span></p> <p>which leaves F, H and J as unknowns and no letters (yet) representing 0, 5 or 9. And these two equations are yet to be solved:</p> <p><span class="math">$$C \times J = AF \Rightarrow 6 \times J = 3F$$</span> : So <span class="math">$$J$$</span> has to be either <span class="math">$$5$$</span> or <span class="math">$$6$$</span> making <span class="math">$$F$$</span> either <span class="math">$$0$$</span> or <span class="math">$$6$$</span>. But we already have a letter for <span class="math">$$6$$</span> (<span class="math">$$C$$</span>), which leaves only the possibility of <span class="math">$$F = 0$$</span> which implies that <span class="math">$$J = 5$$</span>.</p> <p>Which leaves the last equation:</p> <p><span class="math">$$C \times H = JD \Rightarrow 6 \times H = 54 \Rightarrow H = 9$$</span></p> <h1>Table Z</h1> <p>In this table we're back to 9 letters (for some reason the authors decided to miss out on the letter E again!??) and 9 equations:</p> <p><img alt="Table Z" src="/Thoughts/table-z.jpg"></p> <p>As before, from <span class="math">$$H \times G = H$$</span> we can conclude that <span class="math">$$G = 1$$</span>.</p> <p>Plugging <span class="math">$$G = 1$$</span> into the first equation gives:</p> <div class="math">$$H \times J = GJ \Rightarrow H \times J = 10 + J$$</div> <p>... after subtracting <span class="math">$$J$$</span> from both sides of the equation:</p> <div class="math">$$(H-1) \times J = 10$$</div> <p>And since the only factors of 10 is 2 and 5, then one of the following must be true:</p> <ul> <li><span class="math">$$(H-1) = 2$$</span> and <span class="math">$$J = 5$$</span>, i.e. <span class="math">$$H = 3$$</span> and <span class="math">$$J = 5$$</span></li> <li>or <span class="math">$$(H-1) = 5$$</span> and <span class="math">$$J = 2$$</span>, i.e. <span class="math">$$H = 6$$</span> and <span class="math">$$J = 2$$</span></li> </ul> <p>... but we do not yet know which one.</p> <p>We also have the equation <span class="math">$$H \times H = F$$</span> - so <span class="math">$$H$$</span> must be either 0, 2 or 3 (it cannot be 1, since <span class="math">$$G = 1$$</span>). Any higher numbers would result in <span class="math">$$H \times H$$</span> ending up as a two-digit number. And then we can exclude <span class="math">$$H = 0$$</span>, as then we should have expected an <span class="math">$$H$$</span> on the right-hand side of the equation.</p> <p>So it <em>must</em> be the case that <span class="math">$$H = 3$$</span>, as this is the only value which can satisfy <em>both</em> the previous 2 paragraphs!</p> <p>And since <span class="math">$$(H-1) \times J = 10$$</span>, then <span class="math">$$J = 5$$</span></p> <p><span class="math">$$H \times H = F \Rightarrow 3 \times 3 = F \Rightarrow F = 9$$</span></p> <p><span class="math">$$H \times F = BI \Rightarrow 3 \times 9 = BI \Rightarrow B = 2, I = 7$$</span></p> <p><span class="math">$$H \times B = D \Rightarrow 3 \times 2 = D \Rightarrow D = 6$$</span></p> <p><span class="math">$$H \times I = BG \Rightarrow 3 \times 7 = 2G \Rightarrow G = 1$$</span></p> <p><span class="math">$$H \times D = GA \Rightarrow 3 \times 6 = 1A \Rightarrow A = 8$$</span></p> <p><span class="math">$$H \times A = BC \Rightarrow 3 \times 8 = 2C \Rightarrow C = 4$$</span></p> <p>... and before we know it, it is solved!</p> <script type="text/javascript">if (!document.getElementById('mathjaxscript_pelican_#%@#@#')) { var align = "center", indent = "0em", linebreak = "false"; if (false) { align = (screen.width < 768) ? "left" : align; indent = (screen.width < 768) ? "0em" : indent; linebreak = (screen.width < 768) ? 'true' : linebreak; } var mathjaxscript = document.createElement('script'); mathjaxscript.id = 'mathjaxscript_pelican_#%@#@#'; mathjaxscript.type = 'text/javascript'; mathjaxscript.src = 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.3/latest.js?config=TeX-AMS-MML_HTMLorMML'; mathjaxscript[(window.opera ? "innerHTML" : "text")] = "MathJax.Hub.Config({" + " config: ['MMLorHTML.js']," + " TeX: { extensions: ['AMSmath.js','AMSsymbols.js','noErrors.js','noUndefined.js'], equationNumbers: { autoNumber: 'AMS' } }," + " jax: ['input/TeX','input/MathML','output/HTML-CSS']," + " extensions: ['tex2jax.js','mml2jax.js','MathMenu.js','MathZoom.js']," + " displayAlign: '"+ align +"'," + " displayIndent: '"+ indent +"'," + " showMathMenu: true," + " messageStyle: 'normal'," + " tex2jax: { " + " inlineMath: [ ['\\\$$','\\\$$'] ], " + " displayMath: [ ['$$','$$'] ]," + " processEscapes: true," + " preview: 'TeX'," + " }, " + " 'HTML-CSS': { " + " styles: { '.MathJax_Display, .MathJax .mo, .MathJax .mi, .MathJax .mn': {color: 'inherit ! important'} }," + " linebreaks: { automatic: "+ linebreak +", width: '90% container' }," + " }, " + "}); " + "if ('default' !== 'default') {" + "MathJax.Hub.Register.StartupHook('HTML-CSS Jax Ready',function () {" + "var VARIANT = MathJax.OutputJax['HTML-CSS'].FONTDATA.VARIANT;" + "VARIANT['normal'].fonts.unshift('MathJax_default');" + "VARIANT['bold'].fonts.unshift('MathJax_default-bold');" + "VARIANT['italic'].fonts.unshift('MathJax_default-italic');" + "VARIANT['-tex-mathit'].fonts.unshift('MathJax_default-italic');" + "});" + "MathJax.Hub.Register.StartupHook('SVG Jax Ready',function () {" + "var VARIANT = MathJax.OutputJax.SVG.FONTDATA.VARIANT;" + "VARIANT['normal'].fonts.unshift('MathJax_default');" + "VARIANT['bold'].fonts.unshift('MathJax_default-bold');" + "VARIANT['italic'].fonts.unshift('MathJax_default-italic');" + "VARIANT['-tex-mathit'].fonts.unshift('MathJax_default-italic');" + "});" + "}"; (document.body || document.getElementsByTagName('head')).appendChild(mathjaxscript); } </script>I will stop spamming you!2018-09-23T10:30:05+01:002018-09-23T10:30:05+01:00Karl E. J&oslash;rgensentag:None,2018-09-23:/i-will-stop-spamming-you.html<p>The spammers who try to flood my inbox must be getting desperate - they're offering to stop if I pay them!</p> <div class="highlight"><pre><span></span>From: &quot;Admin&quot; &lt;admin@secureserver.eu&gt; Subject: I will stop spaming you I&#39;m sick of sending spam... If you send me $10 USD worth of XMR / Monero I will remove …</pre></div><p>The spammers who try to flood my inbox must be getting desperate - they're offering to stop if I pay them!</p> <div class="highlight"><pre><span></span>From: &quot;Admin&quot; &lt;admin@secureserver.eu&gt; Subject: I will stop spaming you I&#39;m sick of sending spam... If you send me$10 USD worth of XMR / Monero I will remove your address from my list. Address: 84EDuLnccUnGyA8Dqkt1nmAsTy5XWwyMgYgtCtdYYG2C1PcjRyCwcn9eZ9edDL7oqACfVQ19iW9MmJ2MQWfkniUe4Hkygzt Payment ID: u4rjk4uz Thanks Felix </pre></div> <p>I have to laugh - they're so sick of spamming me that they're asking me to help them out of their predicament! I guess just stopping never crossed their mind :-)</p>Blurple!?? Really?2018-09-02T15:18:15+01:002018-09-02T17:04:37+01:00Karl E. J&oslash;rgensentag:None,2018-09-02:/what-is-blurple.html<p>The other day I came across a new English word. This is not a new thing for me - it happens on a annoyingly regular basis. But it made me stop and take notice: BLURPLE !?? Why hadn't i ever heard that before?</p> <p>OK - so it sounds some combination between <a href="https://en.wikipedia.org/wiki/Blue">blue</a> and …</p><p>The other day I came across a new English word. This is not a new thing for me - it happens on a annoyingly regular basis. But it made me stop and take notice: BLURPLE !?? Why hadn't i ever heard that before?</p> <p>OK - so it sounds some combination between <a href="https://en.wikipedia.org/wiki/Blue">blue</a> and <a href="https://en.wikipedia.org/wiki/Purple">purple</a>, right? Pretty much, yes. But does the English language really <em>need</em> a word for that?</p> <p><img alt="Blurple" src="/blurple.svg"></p> <p>At least this is one of the colour names I can <em>understand</em> - the name itself is very descriptive. But it still annoys me that I had not heard of it before!!</p> <p>That is... Until I realised that this was just another world for <a href="https://en.wikipedia.org/wiki/Violet_(color)">violet</a>!! And then <em>that</em> started to annoy me more: What is the point of having two different words (with the exact same meaning) for the same thing!?? Does it mean that whoever invented the word "blurple" did not know the word "violet" ??</p>Suing for Being Ugly!?2018-09-02T00:14:13+01:002018-09-02T21:09:55+01:00Karl E. Jørgensentag:None,2018-09-02:/suing-for-being-ugly.html<p>A Californian woman is allegedly <a class="reference external" href="https://worldnewsdailyreport.com/california-woman-sues-her-parents-because-she-is-ugly/">suing her own parents because she is ugly</a>!</p> <img alt="Ugly woman" class="align-center" src="https://worldnewsdailyreport.com/wp-content/uploads/2017/07/ugly-parents.jpg" /> <p>At least that is what the article claims... It is hilarious and plays into our biases about people from the USA to be litigious and generally a bit weird.</p> <p>Alas, it is fake news. Or to be …</p><p>A Californian woman is allegedly <a class="reference external" href="https://worldnewsdailyreport.com/california-woman-sues-her-parents-because-she-is-ugly/">suing her own parents because she is ugly</a>!</p> <img alt="Ugly woman" class="align-center" src="https://worldnewsdailyreport.com/wp-content/uploads/2017/07/ugly-parents.jpg" /> <p>At least that is what the article claims... It is hilarious and plays into our biases about people from the USA to be litigious and generally a bit weird.</p> <p>Alas, it is fake news. Or to be painfully correct: Satire. On their <a class="reference external" href="https://worldnewsdailyreport.com/disclaimer_/">disclaimer page</a> they do admit:</p> <blockquote> WNDR assumes however all responsibility for the satirical nature of its articles and for the fictional nature of their content. All characters appearing in the articles in this website – even those based on real people – are entirely fictional and any resemblance between them and any persons, living, dead, or undead is purely a miracle.</blockquote> <p>which pretty much gives the game away: It is a satire news web site. just like <a class="reference external" href="https://www.theonion.com/">The Onion</a>: funny if you accept it is satire, horrifying if you take it as fact...</p> <p>But at least it made me smile :-)</p> Recovering From A Fork Bomb2018-08-30T20:26:29+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-30:/recovering-from-a-fork-bomb.html<p>The scenario: Your server is not working right: You cannot start new processes! Simple commands like "ls" fail:</p> <div class="highlight"><pre><span></span>yourname@yourserver:~$ls -bash: fork: retry: Resource temporarily unavailable </pre></div> <p>This is a symptom of the kernel reaching the maximum number of processes. On most systems, this is 32768 processes, and on 64-bit …</p><p>The scenario: Your server is not working right: You cannot start new processes! Simple commands like "ls" fail:</p> <div class="highlight"><pre><span></span>yourname@yourserver:~$ ls -bash: fork: retry: Resource temporarily unavailable </pre></div> <p>This is a symptom of the kernel reaching the maximum number of processes. On most systems, this is 32768 processes, and on 64-bit systems this can be increased to as much as 4194304 (2^22). The current setting is available in <kbd>/proc/sys/kernel/pid_max</kbd>.</p> <p>On a normal system there will be up to a couple of thousand processes running, and reaching 32000 processes will be extremely rare - and almost certainly a clear sign of a problem. (I would wager that this is likely the result of somebody's shell script going recursive or similar)</p> <p>So how do you get the system back to a working state? You will need to stop some processes, and to do that you list to <em>list</em> the processes first. But that will fail too!</p> <div class="highlight"><pre><span></span>yourname@yourserver:~$ps -ef -bash: fork: retry: Resource temporarily unavailable </pre></div> <p>To fix this, you will need to become root. But.. You cannot create a new ssh session to the system (yes: because it is out of process slots). And even <kbd>sudo</kbd> will fail:</p> <div class="highlight"><pre><span></span>yourname@yourserver:~$ sudo -i -bash: fork: retry: Resource temporarily unavailable </pre></div> <p>But there is a way: although you cannot create <em>new</em> processes, you can <em>replace</em> your shell with <kbd>sudo</kbd> using <kbd>exec</kbd>:</p> <div class="highlight"><pre><span></span>yourname@yourserver:~$exec sudo -i [sudo] password for yourname: ******* root@yourserver:~# </pre></div> <blockquote> <p>NOTE: If you get your password wrong, your sudo attempt will be rejected, and <kbd>sudo</kbd> will exit. And you will be instantly logged out (because your shell no longer exists). So don't do that!</p> </blockquote> <p>If the misbehaving process is still running, increasing the maximum number of processes would be futile, as the rogue process would quickly spawn off more processes to reach the new limit. Nobody's typing would be quick enough here...</p> <p>Killing any single one of the rogue processes will free up a process slot, but that slot will quickly be taken up by a new rogue process.. So this is futile too...</p> <p>Even gracefully rebooting the system is unlikely to succeed: To reboot the system, you need to run the shutdown scripts. And the shutdown scripts cannot run since we are out of process slots!</p> <p>One obvious option is to resort to a non-graceful reboot and then hope for the rogue processes are not caused by something that starts upon boot:</p> <div class="highlight"><pre><span></span>root@yourserver:~# reboot --force --force </pre></div> <p>But this is a rather crude option: rebooting systems is frowned upon, especially if people are using it and have not noticed anything wrong. And rebooting this way increases the risk of data corruption to uncomfortable levels.</p> <p>It <em>is</em> possible to recover the system to a working state <em>without</em> resorting to desperate measures. But it is tricky, as we have to work within some rather severe constraints:</p> <ul> <li>We cannot execute any commands that require new processes. This limits us to using shell built-in commands.</li> <li>The rogue processes will re-spawn processes as soon as one is killed. So killing them one at a time is not an option.</li> </ul> <p>The first problem is listing processes. This can be accomplished using only shell builtins and the <kbd>/proc</kbd> file system:</p> <div class="highlight"><pre><span></span>cd /proc; for pid in [0-9]*; do test -r$pid/exe &amp;&amp; read pid comm state ppid pgrp session tty_nr tpgid junk &lt; $pid/stat &amp;&amp; read loginuid &lt;$pid/loginuid ; printf &#39;%5d %5d %-25s %-10s %5d %5d %5d %5d\n&#39; $pid$ppid $comm$loginuid $pgrp$session $tty_nr$tpgid; done </pre></div> <p>This will give a somewhat basic equivalent to ps(1) and should allow us to spot the name of the rogue processes.</p> <p>The next step is to <em>freeze</em> the rogue processes by sending the STOP signal to them. This will prevent them from spawning new child processes once process slots become available: Replace "<kbd>(rogue-process)</kbd>" with the name of the process:</p> <div class="highlight"><pre><span></span>cd /proc; for x in 1 2 3 4 5; do for pid in [0-9]*; do test -r $pid/exe &amp;&amp; read pid name state ppid junk &lt;$pid/stat &amp;&amp; [ $name = &#39;(rogue-process)&#39; ] &amp;&amp; kill -STOP$pid; done; done </pre></div> <p>Repeat the above multiple time if you find multiple binaries being responsible: you need to make sure you get them all.</p> <blockquote> <p>Make sure that you do <em>not</em> freeze the processes you are using - i.e. your shell, ssh daemon etc. If you do that, you are dead in the water...</p> </blockquote> <p>Once the rogue processes have been frozen, we can get away with killing them one at a time as they cannot respawn stuff (again: Replace "<kbd>(rogue-process)</kbd>" with the name of the rogue process):</p> <div class="highlight"><pre><span></span>cd /proc; for pid in [0-9]*; do test -r $pid/exe &amp;&amp; read pid name state ppid junk &lt;$pid/stat &amp;&amp; [ $name = &#39;(rogue-process)&#39; ] &amp;&amp; kill -KILL$pid; done </pre></div> <p>and you <em>should</em> be back to normal!</p>Files Hidden By Mount Points2018-08-28T22:51:50+01:002018-09-01T14:19:01+01:00Karl E. J&oslash;rgensentag:None,2018-08-28:/files-hidden-by-mount-points.html<p>The wolderful flexibility of being able to mount file systems at any point in the file system comes at a small cost: You can end up <em>hiding</em> files - usually by mistake.</p> <p>For example, imaging having the following file systems:</p> <ul> <li><kbd>/</kbd> - the root file system. A small(ish) partition, usually only a …</li></ul><p>The wolderful flexibility of being able to mount file systems at any point in the file system comes at a small cost: You can end up <em>hiding</em> files - usually by mistake.</p> <p>For example, imaging having the following file systems:</p> <ul> <li><kbd>/</kbd> - the root file system. A small(ish) partition, usually only a few gigabytes in size.</li> <li><kdb>/home</kbd> - for people's <kbd>$HOME</kbd> directories.</li> </ul> <p>Pretty normal. You probably have other mountpoints too. But things can be hidden simply by mounting things: Imagine this scenario:</p> <ul> <li>Unmount <kbd>/home</kbd></li> <li>Create some big files in <kbd>/home</kbd>. These will end up in the <em>root</em> file system, and consume space from here.</li> <li>Re-mount <kbd>/home</kbd></li> </ul> <p>The big files that you created in <kbd>/home</kbd> are no longer accessible because <kbd>/home</kbd> now refers to a <em>different</em> file system. The files will still be consuming disk space, and unmounting <kbd>/home</kbd> will make them re-appear.</p> <p>The stupid and simple way to track down such files involves simply unmounting the file systems in question and having a look. But this is not always possible on a system that is in use: unmounting the file systems means making stuff unavailable and stopping processes...</p> <p>Here the linux kernel concept of <a href="https://en.wikipedia.org/wiki/Linux_namespaces">namespaces</a> come to the rescue: it is possible for different processes to see different mount points. By default, a process belongs to the same namespace as its parent process, and 99% of the time, this is what we want. Except now.</p> <p>In order to investigate files hidden by mount points, we simply start a new process in its <em>own mount namespace</em>, which is independent of the default (parent) namespace. In this namespace we can unmount things at will (because it will only affect our own process), so we can "peek" under the mount points:</p> <div class="highlight"><pre><span></span># unshare --mount # This will start a new interactive subshell # umount /home # Only affects our own namespace # ls -l /home # exit # exit the subshell created by unshare </pre></div> <p>Voila!</p>Tracking Down Deleted-But-Still-Open Files2018-08-25T16:16:22+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-25:/tracking-down-deleted-but-still-open-files.html<p>The situation: <kbd>df</kbd> (which shows disk usage) disagrees with <kbd>du</kbd> (which shows disk space taken up by files). So you have disk space which is "used", but it is not taken up by any files!? What gives!?</p> <p>For example:</p> <div class="highlight"><pre><span></span>root@xps:~# df -h /tmp Filesystem Size Used Avail Use% Mounted …</pre></div><p>The situation: <kbd>df</kbd> (which shows disk usage) disagrees with <kbd>du</kbd> (which shows disk space taken up by files). So you have disk space which is "used", but it is not taken up by any files!? What gives!?</p> <p>For example:</p> <div class="highlight"><pre><span></span>root@xps:~# df -h /tmp Filesystem Size Used Avail Use% Mounted on /dev/mapper/xps--vg-tmp 4.9G 4.6G 0 100% /tmp root@xps:~# du -sh /tmp 708K /tmp </pre></div> <p>Here we have a discrepancy of at least 21 Mb - more than can be accounted for by <a href="/my-disk-space-doesnt-add-up.html">filesystem overhead</a>. So there has to be another explanation.</p> <p>If <kbd>df</kbd> and <kbd>du</kbd> ran at different times on an active system, it is possible that things changed in between those times. So it is usually worth double-checking that you are looking at up-to-date information before jumping to conclusions. But since you are still reading this, let us assume this that the information is up-to-date...</p> <p><kbd>du</kbd> and <kbd>df</kbd> can disagree due to <em>open files which have been deleted</em>: Linux allows files to be deleted while they are still open (the disk space allocated to the file will not be returned to the "free" pool until the file is closed) This makes the file "invisible" to <kbd>du</kbd> (because it no longer appears in any directory), but <kbd>df</kbd> shows the disk space as occupied.</p> <p>So how do we find such files? They will <em>not</em> appear in any directory because they have been deleted, so looking for them in the file system is futile.</p> <p>Here <kbd>lsof</kbd> (in the Debian package of the same name) comes to the rescue. </p> <div class="highlight"><pre><span></span>lsof /tmp | grep deleted | sort -n -k7 -r </pre></div> <p>This might give output looking like this:</p> <div class="highlight"><pre><span></span>bash 11164 karl 4w REG 253,12 4909318144 57 /tmp/somefile (deleted) firefox-e 31391 karl 144r REG 253,12 939693 46 /tmp/tmp-blm.xpi (deleted) gnome-ter 24045 karl 15u REG 253,12 262144 50 /tmp/#50 (deleted) gnome-ter 24045 karl 23u REG 253,12 196608 60 /tmp/#60 (deleted) gnome-ter 24045 karl 20u REG 253,12 196608 21 /tmp/#21 (deleted) gnome-ter 24045 karl 17u REG 253,12 65536 61 /tmp/#61 (deleted) gnome-ter 24045 karl 16u REG 253,12 65536 58 /tmp/#58 (deleted) apache2 7264 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 7263 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 7262 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 7261 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 7260 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 7259 www-data 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) apache2 1338 root 9u REG 253,12 0 42 /tmp/.ZendSem.Agz5mK (deleted) </pre></div> <p>This lists the process which has the deleted files open - in this case, the disk space was being used by <kbd>/tmp/somefile</kbd> which was open by process id <kbd>11164</kbd>. So stopping/killing that process would free up the disk space.</p>When Sorting Fails2018-08-24T16:52:55+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-24:/when-sorting-fails.html<p>When investigating disk usage issues on a very full disk, the investigation may actually be hampered by the lack of disk space. For example:</p> <div class="highlight"><pre><span></span>du -x -m / | sort -nr | head -25 </pre></div> <p>can fail if you have a lot of directories and your disk is completely full: <kbd>sort</kbd> will only process …</p><p>When investigating disk usage issues on a very full disk, the investigation may actually be hampered by the lack of disk space. For example:</p> <div class="highlight"><pre><span></span>du -x -m / | sort -nr | head -25 </pre></div> <p>can fail if you have a lot of directories and your disk is completely full: <kbd>sort</kbd> will only process "so much" before it resorts using a temporary file, and it cannot create a temporary file because the disk is full!</p> <p>Here we need to resort to an often overlooked option on the <kbd>sort</kbd> command: <kbd>--buffer-size</kbd>: this will force it to allocate a memory buffer of the given size - just make sure that the buffer is big enough to contain all of the output from <kbd>du</kbd>:</p> <div class="highlight"><pre><span></span>du -x -m / | sort -nr --buffer-size=50M | head -25 </pre></div> <p>50 Mb should be enough for most cases. If your server also so short on memory that this is impossible: You're <a href="https://www.urbandictionary.com/define.php?term=SOL">SOL</a>!</p> <p>Another way of working around this is to direct the <kbd>sort</kbd> command to place its temporary files somewhere else, by setting the <kbd>TMPDIR</kbd> environment variable:</p> <div class="highlight"><pre><span></span>export TMPDIR=/path/to/dir du -x -m / | sort -nr | head -25 </pre></div> <p>But this assumes that you have some free diskspace <em>somewhere</em>...</p>My Disk Space Doesn't Add Up!2018-08-24T11:21:21+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-24:/my-disk-space-doesnt-add-up.html<p>Sometimes your disk usage does not add up - for example:</p> <div class="highlight"><pre><span></span>karl@someserver:~$ df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda1 52G 49G 243M 100% / </pre></div> <p>So what gives? 49Gb + 248Mb is definitely <em>less than 52Gb</em> !! This is a sizeable discrepancy of at least 1.5 Gb!</p> <p>Also, the …</p><p>Sometimes your disk usage does not add up - for example:</p> <div class="highlight"><pre><span></span>karl@someserver:~\$ df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda1 52G 49G 243M 100% / </pre></div> <p>So what gives? 49Gb + 248Mb is definitely <em>less than 52Gb</em> !! This is a sizeable discrepancy of at least 1.5 Gb!</p> <p>Also, the percentage looks off: 49/52 =~ 94.2% - which is definitely not 100%!</p> <p>The answer is simple: <em>filesystem overhead</em> : If you have a disk device of 52Gb, and put a file system on it, do not expect to write a 52Gb file to the resulting file system. By necessity, the file system itself will need disk space for:</p> <ul> <li>keeping track of which disk blocks are free (or bad)</li> <li>directory listings</li> <li>file and directory metadata (ownership, permissions, modification times etc)</li> <li>user disk quota management</li> <li>journal - for crash recovery</li> <li>space reserved for the <kbd>root</kbd> user - usually 5% (this is not included in the "used" amount)</li> </ul> <p>The above is true for the most commonly used file system: ext4. Other file systems vary, but they will <em>all</em> have some overhead.</p>Cleaning Up Disk Space2018-08-23T22:53:43+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-23:/cleaning-up-disk-space.html<p>Let us assume you already figured out <a href="/where-did-my-disk-space-go.html">where your disk space went</a>.</p> <p>Cleaning up disk space is easy: You just delete stuf, right?</p> <p>Well.. Mostly. There are a couple of things to watch out for:</p> <ul> <li>Deleting important stuff is ... bad. You still want a working system, right?</li> <li>Deleting files that …</li></ul><p>Let us assume you already figured out <a href="/where-did-my-disk-space-go.html">where your disk space went</a>.</p> <p>Cleaning up disk space is easy: You just delete stuf, right?</p> <p>Well.. Mostly. There are a couple of things to watch out for:</p> <ul> <li>Deleting important stuff is ... bad. You still want a working system, right?</li> <li>Deleting files that are in use will <em>not</em> free up the disk space until processes close the files.</li> </ul> <h2>Cleaning Out Log Files</h2> <p>Often, the culprit will be a log file that has grown beyond expectations: so you have two options:</p> <ul> <li> <p>Stop the process writing to the log file, remove the log file, and start the process again. Obviously this implies that some (important?) service will be down for a short time. You will have to decide whether the downtime is acceptable.</p> </li> <li> <p>Simply truncate the log file with the <em>smiley of death</em>:</p> <div class="highlight"><pre><span></span>:&gt; /path/to/offending-file </pre></div> <p>This makes use of the <em>colon</em> shell command - which is a no-op. Some people prefer to use the much-long-to-type command:</p> <div class="highlight"><pre><span></span>echo &gt; /path/to/offending-file </pre></div> <p>which not only is much longer to type, but also puts a newline in the file. Why bother!?</p> </li> </ul>Where Did My Disk Space Go?2018-08-23T22:53:43+01:002018-08-31T19:23:54+01:00Karl E. J&oslash;rgensentag:None,2018-08-23:/where-did-my-disk-space-go.html<p>Imagine the scenario: Your application does not work, and no errors are appearing in your log files. In fact: <em>nothing</em> new is appearing in your log files! Bah!</p> <p>And then you discover that your root file system is full. And that's the only file system on the box:</p> <div class="highlight"><pre><span></span>Filesystem 1K-blocks …</pre></div><p>Imagine the scenario: Your application does not work, and no errors are appearing in your log files. In fact: <em>nothing</em> new is appearing in your log files! Bah!</p> <p>And then you discover that your root file system is full. And that's the only file system on the box:</p> <div class="highlight"><pre><span></span>Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 18451132 18451132 0 100% / </pre></div> <p>At the very least, that explains the absense of new log entries and probably also why your application is acting up.</p> <p>Note that the numbers <a href="/my-disk-space-doesnt-add-up.html">may not add up exactly</a>. Let's ignore that for now.</p> <p>So.. where did the disk space go?</p> <p>The quick and simple way of finding out is to use the <kbd>du</kbd> command and process the output so you get the biggest hierachies at the top:</p> <div class="highlight"><pre><span></span>du -x -m / | sort -nr | head -25 </pre></div> <p>Note: the <kbd>sort</kbd> command may fail due to lack of disk space - but there are <a href="/when-sorting-fails.html">ways around that</a>.</p> <p>One thing to look out for: The total disk usage of <kbd>/</kbd> should be close to the size of the file system. If not, then there are 3 possible reasons:</p> <ul> <li><a href="/my-disk-space-doesnt-add-up.html">filesystem overhead</a></li> <li>deleted (but still open) files. There are <a href="/tracking-down-deleted-but-still-open-files.html">ways of hunting these down</a></li> <li>files <a href="/files-hidden-by-mount-points.html">hidden by a mount point</a></li> </ul> <p>So what to do about it? Several things:</p> <ul> <li> <p>You probably need to <a href="/cleaning-up-disk-space.html">clean up disk space</a> right away</p> </li> <li> <p>Consider <a href="/drafts/adding-disk-space.html">adding disk space</a></p> </li> <li> <p>How did the situation get this bad? You need <em>monitoring</em>, so you can be alerted before things get <em>this</em> bad! Seriously!</p> </li> <li> <p>Find out what caused the growth in disk usage: Was is necessary? Is there bug that needs to be resolved?</p> </li> </ul>