5 private links
This paper by Leiserson et. al (the L from the famous CLRS Algorithms book) explores how computer performance can continue to improve after Moore's law hits its limits. In this paper from 2020, the authors predict that there will be diminishing returns after 5 nm transistors, but there are 3 nm and 2 nm transistors being manufactured in 2024 (not available to China though they have the highest number of supercomputers/country). Clock speed had pretty much stopped increasing since 2004 due to increasing power requirements. We cannot continue to reduce voltage with increasing clock frequency (Dennard scaling).
The gains at the bottom are due to improvements in semiconductor technology - the tide that lifts all boats. The gains at the top on the other hand come from software performance optimization (while taking the target hardware into account), algorithmic efficiency and hardware streamlining.
Unlike the historical gains at the Bottom, however, the gains at the Top will be opportunistic, uneven, sporadic, and subject to diminishing returns as problems become better explored.
Why programming languages don't provide graphs in their standard libraries:
- There are many different kinds of graphs
- There are many different representations of each kind of graph
- There are many different graph algorithms
- Graph algorithm performance is very sensitive to graph representation and implementation details
- People run very expensive algorithms on very big graphs.
This article from 1995 is just as relevant today. Just convert the KBs to MBs and MBs to GBs.
Some of the author's identified causes of software bloat:
- user-friendliness (with graphical elements)
- measuring power of software by its number of features
- misinterpreting complexity as sophistication
- profitability of customer support contracts
- lack of interest in refining existing software
- time pressure to ship faster than one's competitors
- using C instead of languages with good abstractions (i.e. object-oriented ones)
Niklas Wirth was involved in the design and implementation of a new system called Oberon (using a programming language derived from Modula-2) in hardware, meant for worstations.
Three underlying tenets of the Oberon system
- Concentrate on the essentials
- Use a type-safe object-oriented programming language
- Extensible using new data types
Oberon's core is only 200 Kilobytes, including the editor and compiler. It loads modules on demand and every module is in memory at most once.
This was achieved by carefully designing the modules and iteratively refactoring them.
9 lessons learnt building Oberon:
- Use a strongly typed language
- Module hierarchy design with the least duplication
- Type extension
- Identify primitives and avoid their proliferation
- Designing a system that can be understood in its entirety by an individual
- Keep the team size small to avoid communication problems
- Try to keep complexity low at every step of the process
- Programmers participating in all aspects of the development and dogfooding
- Not publishing programs until they are of publication quality
Some of these patterns could also be useful in other applications.