text:
# ATLAS FAQ

[Home] [Docs] [FAQ] [Errata] [Software] [Install] [Support] [Lists] [Developer home] [Timings]

ATLAS BASICS:

INSTALL QUESTIONS:

POST-INSTALL QUESTIONS:

[Home] [Docs] [FAQ] [Errata] [Software] [Install] [Support] [Lists] [Developer home] [Timings]

### What is ATLAS?

ATLAS stands for Automatically Tuned Linear Algebra Software. ATLAS is both a research project and a software package. This FAQ describes the software package. ATLAS's purpose is to provide portably optimal linear algebra software. The current version provides a complete BLAS API (for both C and Fortran77), and a very small subset of the LAPACK API. For all supported operations, ATLAS achieves performance on par with machine-specific tuned libraries.

### Who uses ATLAS?

ATLAS can be used by anyone needing fast linear algebra routines. ATLAS is used directly by a great many research scientists. Because of the open nature of ATLAS, we have no way of knowing how many users of ATLAS there are. In the following paragraphs, we indicate some of the users that we know about, but this is far from a complete list.

ATLAS is used, or is planned to be used, in the following PSEs:

ATLAS is also included in: ATLAS may be optionally used by almost any project requiring the BLAS. Here are some projects that we have seen providing the option for using ATLAS:

Additionally, ATLAS is included in some way by the following OS distributions:

## What are the academic references for ATLAS?

The academic references for ATLAS are given in bibtex format below. If you want to reference one paper only, probably the newest (first shown) is the best, as it references the others. The first two papers contain the bulk of the needed information. Referencing the homepage can help other researchers find the software.

Note that there have been quite a few subsequent papers that discuss ATLAS (with varying degrees of accuracy and detail) written by people not directly involved in ATLAS's production and design. While these papers may be about ATLAS, they are not, obviously, primary sources, and should not be cited as such. If the paper is not authored by Whaley or Petitet, it is not a primary-source ATLAS paper.

@ARTICLE{whaley04,
AUTHOR = "R. Clint Whaley and Antoine Petitet",
TITLE  = "Minimizing development and maintenance costs in supporting
persistently optimized {BLAS}",
JOURNAL= "Software: Practice and Experience",
volume = "35",
number = "2",
pages  = "101-121",
month  = "February",
YEAR   = "2005",
NOTE           = {\verb+http://www.cs.utsa.edu/~whaley/papers/spercw04.ps+}
}

@ARTICLE{WN147,
AUTHOR         = "R. Clint Whaley and Antoine Petitet and Jack J. Dongarra",
TITLE          = "Automated Empirical Optimization of Software and the
{ATLAS} Project",
JOURNAL        = "Parallel Computing",
VOLUME         = "27",
NUMBER         = "1--2",
PAGES          = "3--35",
YEAR           = 2001,
NOTE           = "Also available as University of Tennessee LAPACK Working
Note \#147, UT-CS-00-448, 2000
({\tt www.netlib.org/lapack/lawns/lawn147.ps})" }

@inproceedings{atlas_siam,
AUTHOR         = {R. Clint Whaley and Jack Dongarra},
TITLE          = "{Automatically Tuned Linear Algebra Software}",
BOOKTITLE      = "Ninth SIAM Conference on Parallel Processing for
Scientific Computing",
NOTE           = "CD-ROM Proceedings",
YEAR           = 1999 }

@inproceedings{atlas_sc98,
AUTHOR    = "R. Clint Whaley and Jack Dongarra",
TITLE     = "Automatically Tuned Linear Algebra Software",
BOOKTITLE = "SuperComputing 1998: High Performance Networking and Computing",
YEAR      = "1998",
NOTE      = "CD-ROM Proceedings. {\bf Winner, best paper in the systems
category.}\\
URL: \verb+http://www.cs.utsa.edu/~whaley/papers/atlas_sc98.ps+"
}

@techreport{atlas_wn97,
AUTHOR         = {R. Clint Whaley and Jack Dongarra},
TITLE          = "{Automatically Tuned Linear Algebra Software}",
INSTITUTION    = "University of Tennessee",
YEAR           = "1997",
MONTH          = "December",
NUMBER         = "UT-CS-97-366",
NOTE           = "URL : \verb+http://www.netlib.org/lapack/lawns/lawn131.ps+"
}

@UNPUBLISHED{atlas-hp,
TITLE = "ATLAS homepage",
AUTHOR = "{See homepage for details}",
NOTE   = "http://math-atlas.sourceforge.net/"
}

### Does ATLAS run on my platform (OS/hardware)?

ATLAS should produce optimized libraries on almost any platform possessing an ANSI/ISO C compiler, and some Unix-like command-line tools (eg., make, cp, etc). ATLAS runs on pretty much all Unix variants (including embedded systems), as well as Windows (Windows users must install the free cygnus tools).

### What software license does ATLAS use (AKA: in what ways and for what purposes am I allowed to use ATLAS)?

ATLAS uses a BSD-style license, without the advertising clause. ATLAS's license is taken almost verbatim from the example given at opensource.org. Here is the relevant portion of the license, as taken from an ATLAS source file:
 * Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*   1. Redistributions of source code must retain the above copyright
*      notice, this list of conditions and the following disclaimer.
*   2. Redistributions in binary form must reproduce the above copyright
*      notice, this list of conditions, and the following disclaimer in the
*      documentation and/or other materials provided with the distribution.
*   3. The name of the ATLAS group or the names of its contributers may
*      not be used to endorse or promote products derived from this
*      software without specific written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE ATLAS GROUP OR ITS CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.

To see the exact license, simply edit almost any source file in the ATLAS tarfile (eg., ATLAS/src/auxil/ATL_lcm.c).

### How do I get help/technical support with ATLAS?

Your first resource for should always be the ATLAS errata file. This file keeps track of all discovered errors in ATLAS, and their workarounds or fixes. It also contains workarounds for common system problems (eg., compiler errors, non-standard commands, etc), as well containing advice necessary to get the best performance on various machines.

If you have downloaded the ATLAS source, your ATLAS/doc directory contains some useful documentation, though it is often more dated than the info in the errata and online.

If (and only if) neither of these sources provides the information you need, you can can submit a support request to:

Do not, under any circumstances, post your support request to the "bug" tracker. As documented on the tracker itself, this is for developer confirmed bugs only. All users should use the support or feature request trackers. Things that turn out to be bugs will later be escalated to the bug tracker by the confirming developer.

You do not need to create SourceForge account in order to use the tracker (that persistant plea to "please log in" can be ignored), though it makes things easier if you do. In particular, if you don't log in, you won't be able to later attach extra files, etc (you can attach a file in your initial report, but afterwords, it is unsure you are the original poster, so it won't allow it). So, if you think you may need to do this kind of thing relatively often, it may be worth doing.

Note that you should upload the error_[ARCH].tgz file as well. If the error killed the ATLAS install before it succesfully created the error tarfile, create it yourself by issuing the following command from your BLDdir subdirectory:

   make error_report

Note that the [ARCH] of the above directions should be replaced by your architecture string that ATLAS is using (eg., Linux_P4SSE1 or SunOS_SunUS4, etc).

### What documentation is available (usage info)?

ATLAS's main job is to provide optimized libraries, so most of the documentation is on the appropriate APIs. ATLAS does provide some executables, but these are merely testers and timers for the provided libraries. A very rough description of the operation of these executables is given in ATLAS/doc/TestTime.txt in your ATLAS source directory.

Here's some pointers to ATLAS documentation:

ATLAS stable errata
Errata file for last stable release. Includes many architecture-specific install hints, as well as documenting and providing fixes for all known errors.
ATLAS installation guide
Guide to configuring, installing, and testing (accuracy and efficiency) ATLAS. Also available in the tarfile at ATLAS/doc/atlas_install.pdf.
ATLAS FAQ
ANSI/ISO C BLAS API reference
A brief API reference for the ANSI/ISO C interface to the BLAS.
Fortran77 BLAS API reference
A brief API reference for the Fortran77 interface to the BLAS.
Fortran77 and C LAPACK API reference
A brief API reference for the LAPACK routines provided by ATLAS.
Contributer HOWTO [ps] [html]
Paper describing mechanism to speedup ATLAS by contributing kernels.
Developer HOWTO [ps] [html]
Paper with information on how to be an ATLAS developer.
LAPACK homepage
All kinds of LAPACK-related info given at the LAPACK homepage.
BLAS homepage
All kinds of BLAS-related info given at the BLAS homepage.

### What mailing lists, archives, and so on does ATLAS have?

ATLAS has the following tracker lists:
ATLAS technical support
The place you go once you are sure you have a technical support issue not covered in the other docs. Anyone can post to this list.
ATLAS feature request list
The place to request additions/extensions to ATLAS. Anyone can post to this list.

ATLAS also has various mail lists and archives. Anyone can sign up or post to these guys. They are:

ATLAS announcement list [browse] [subscribe/unsubscribe/preferences]
Low-volumne list, where significant changes in stable release are announced. Mainly used to announce new stable releases, perhaps occasional updates about very important features appearing in developers' release. Traffic should be O(10) messages a year.
ATLAS error list [browse] [subscribe/unsubscribe/preferences]
Low traffic list for errors found in the ATLAS stable release. Errors in config or install portions are generally not reported, but those involving getting incorrect results or substantial performance penalties are. The idea is that the people monitoring this list are using a successfully installed ATLAS library, and want to be informed if errors are discovered.
ATLAS developer list [browse] [subscribe/unsubscribe/preferences]
Relatively high volume list for ATLAS developer communication.
ATLAS Performance/results list [browse] [subscribe/unsubscribe/preferences]
Mailing list for reporting performance results from ATLAS.
ATLAS CVS commits list [browse] [subscribe/unsubscribe/preferences]
High volume traffic list, with a message for each CVS commit.

Unfortunately, we lack the manpower to provide prebuild binaries.

### Can I get ATLAS in rpm or .deb or some other format?

Our only supported format is a compressed tarfile. If you really feel the need for .rpm or .deb versions, other parties (eg, Debian, SuSE) provide them (note that we can't answer questions on ATLAS installed in this way, however, since we don't know much about them). ATLAS provided by third parties may not be as up-to-date, or may run slower than compiled ATLAS (eg., some companies compile only a couple of x86 libraries, so that they would use the same library for the P4 and P4E chips, even though ATLAS should tune itself separately to all the x86 variants for maximal performance).

### What does the version number of ATLAS mean?

ATLAS version numbers look like: <major number>.<minor number>.<update number>. The meaning of these terms is:
Major number
Major release numbers are changed only when fairly large, sweeping changes are made. Changes in the API are the most likely to cause a major release number to incremement. For example, when ATLAS went from supporting only matrix multiply to all the Level 3 BLAS, the major number changed; the same happened when ATLAS went from supporting only Level 3 BLAS to all BLAS.
Minor number
Minor release numbers are changed at each official release. Even numbers represent stable releases, while odd minor numbers are reserved for developer releases. Click here for an explanation of developer and stable releases.
Update number
Update numbers are essentially patches on a particular release. For instance, stable ATLAS releases only occur roughly once per year. As errors are discovered, they are errataed, so that a user can apply the fixes by hand. When enough errata are built up that it becomes impractical to apply the important ones by hand, we will issue an update. So, stable updates are typically bug fixes, or important system workarounds, while developer releases often involve new code. A typical number of updates to a stable release might be something like 4. By their very nature, each update to the developer release usually contains new code, and they can happen relatively rapidly. A developer release may have any number of updates.

So, 3.2.1 would be a stable release, with one group of fixes already applied. 3.3.12 would be the 12th update (13th release) of the associated developer release.

### How can I tell what version of ATLAS I have?

For ATLAS version 3.3.6 or newer, you can find out version and build information via the routine ATL_buildinfo. The following complete program will give build information (including version number) when linked against version 3.3.6 or later libatlas.a's:
main()
/*
* Compile, link and run with something like:
*    gcc -o xprint_buildinfo -L[ATLAS lib dir] -latlas ; ./xprint_buildinfo
* if link fails, you are using ATLAS version older than 3.3.6.
*/
{
void ATL_buildinfo(void);
ATL_buildinfo();
exit(0);
}

If you are using an ATLAS version prior to 3.3.6, there is no easy way to find the version information without looking at the source. If you have the source tree around, the easiest fix is to examine pretty much any source file (eg. ATLAS/src/auxil/ATL_lcm.c); the major and minor version number will be given in the copyright notice at the top. To find out the update number, you'd have to consult the actual routines updated by the particular update, as given in the ATLAS errata file.

If you still have the directory where you built ATLAS around, you can find this version information w/o writing the above routine by:

   cd BLDdir/bin
make xprint_buildinfo
./xprint_buildinfo

### What's the difference between stable and developer ATLAS releases?

The vast majority of ATLAS users should download and use only the stable version of ATLAS. Stable versions of ATLAS are released roughly once a year. The most current stable release has an associated errata file, which details all errors found in the release. Stable releases are very well tested, and are much more heavily supported.

Developer releases, on the other hand, are meant to be used, as the name suggests, by ATLAS developers, contributers, and people happy to live on the bleeding edge. Developer releases are meant to allow access to the newest ATLAS sources, and may represent a simple snapshot of the internal developer tree. As such, they are essentially untested, and may not build, much less run, correctly. So, while they may possess features not available in the current ATLAS release, only the most experienced of users should consider utilizing them.

Developer releases are available from the developer site, while stable releases are available from the ATLAS main page. Stable and developer releases are also distinguished by their version numbers, as explained here.

### What LAPACK routines does ATLAS provide?

The only way to be sure you have the most up-to-date list is to examine the source in ATLAS/interfaces/lapack/F77/src/. It is pretty much a foregone conclusion that any documentation, this page included, will eventually become out of date. ATLAS 3.6 and 3.8 provide C and Fortran77 interfaces to these routines:
• [S,D,C,Z]GESV
• [S,D,C,Z]GETRF
• [S,D,C,Z]GETRS
• [S,D,C,Z]GETRI
• [S,D,C,Z]TRTRI
• [S,D,C,Z]POSV
• [S,D,C,Z]POTRF
• [S,D,C,Z]POTRS
• [S,D,C,Z]POTRI
• [S,D,C,Z]LAUUM

Since LAPACK has no official C API, ATLAS provides its own in ATLAS/interfaces/lapack/C/src/.

### What header files does ATLAS provide?

The official header file for the C interface to the BLAS is available as ATLAS/include/cblas.h. The header file for the C interface to LAPACK is ATLAS/include/clapack.h.

### How can I get dynamic (.so) libraries rather than ATLAS's default static libraries (.a)?

ATLAS 3.8.0 has prototype support for building dynamic libraries, as described here.

### What's the best hardware for running ATLAS/what machine do you recommend I buy for this kind of work?

This is another question that is pretty much impossible to answer generally or keep up to date. I need to update this entry once I get my hands on the new Opteron!

### Can I use ATLAS with CLAPACK?

Yes. CLAPACK gives you the option to compile CLAPACK to use the standard C interface to the BLAS, which ATLAS provides. If you run CLAPACK's included BLAS tester, be sure to turn off error-exit tests, since it can't properly test the error exits returned by the CBLAS. ATLAS provides essentially the same testers in ATLAS/interfaces/blas/C/testing, which do correctly test the error exits, if that's important to you.

### How well optimized are the various routines in ATLAS?

All of the routines in ATLAS tend to be competitive with the machine-specific versions for most known architectures. However, ATLAS is not just about working well on known architectures, but also tries to be optimal for unknown machines. When it comes to the generality of the optimizations ATLAS uses, there is a definite heirarchy:
• Level 3 BLAS are very general in their optimizations, and will be competitive on almost all cache-based architectures possessing a decent ANSI/ISO C compiler.
• The provided LAPACK routines utilize a recursive algorithm that should yield reliably better results than the more common staticly-blocked algorithms.
• The Level 1 and 2 BLAS are less general, and may not perform as well on significantly different architectures (eg., IA64).

### I need routine/architecture X optimized, can you do it?

If you have a particular operation and/or architecture you really need optimized, you may want to post a mention of that to the ATLAS feature request tracker. We don't do optimization on request, but when we have to choose the next set of operations to support, user input can certainly influence things. To maximize your chance of swaying us, you'll want to include what percentage of your application time is spent in the particular operation, etc.

A quicker way to get action is to do it yourself. ATLAS is open source, and The developer homepage explains how you can use ATLAS to optimize various operations.

### How is ATLAS funded?

ATLAS is presently funded by NSF CAREER OCI-1149303. It was supported in the past by NSF CRI CNS-0551504 and the NSF EPSCoR Cooperative Agreement No. EPS-1003897, with additional support from the Looisiana Board of Regents. For more details, see here.

I originally started ATLAS development when I worked at the Innovative Computer Laboratory at the University of Tennessee. I got enough of it working to convince Jack (Dongarra, of LAPACK and BLAS fame) to give the development go-ahead on my own time. After that, ATLAS was written into a variety of grants, but was never funded (to my knowledge) solely on its own grant at ICL. I believe some of it's later development took place under the NSF grant "Linear Algebra Algorithms and Tools for Emerging Computing Environments and User Communities", Grant Number ACI-9813362.

Both Antoine and myself (the two full-time ATLAS researchers and developers) left ICL in 2001. After this date, ATLAS work was pretty much entirely unsupported, which slowed development in a massive way. However, in 2003, Advanced Micro Devices funded a year of my graduate studies in return for some Opteron tuning. This allowed me to spend quite a bit more time on ATLAS than previously, and resulted in the release of ATLAS 3.6.0 in that year.

After this, I found very little time to work on ATLAS due to faculty duties until 2006 when I got funding for both research and maintainence, thanks to both NSF and DoD. Details can be found here. ATLAS development has therefore picked up again, and ATLAS 3.8.0 was released in October of 2007.

At present, the main support for ATLAS comes from my CAREER award. The government contract was not transfered to LSU, but Tony Castaldo is supported on it until mid-2014.

### Who provides infrastructure support?

Obviously, Sourceforge provides the the ATLAS main page, including CVS services, tracker, etc. Also netlib provides access for a large part of the mathematical community through ATLAS's original homepage.

As far as machine access for tuning:

• Apple has provided local G4, G5, CoreDuo, and Core2Duo access for ATLAS development.
• Kate Minola at the University of Maryland, College Park, provided access to a wide array of machines for the most ATLAS 3.8.0, including providing my sole access to solaris/x86 and AIX/POWER4.
• I have access to a wide variety of architectures through my old colleagues at ICL.
• Advanced Micro Devices provided local Opteron access for ATLAS3.6 development.
• I've used machines through Sourceforge's compilefarm.

### Who wrote/contributed to ATLAS?

Note that this question addresses package design and code contribution, not money, infrastructure or testing. R. Clint Whaley founded the ATLAS project. After the initial release, he was joined on the project by Antoine Petitet. Between them, these two individuals are responsible for 95% of the code in ATLAS, along with pretty much all of the design. That is not to say that others have not made substantial contributions, however.

In particular, ATLAS has been designed to allow for outside contribution such that a user can provide only a very small kernel, and thus speed up large portions of the library. Many people have contributed in this manner, and this has resulted in extremely large performance improvements for ATLAS on certain architectures. These contributers (in alphabetic order), and a rough sketch of what they have done, are:

Doug Aberdeen
His work on emmerald (an SSE-enabled SGEMM), and help on atlas developer mailing list, was the starting point for a lot of people working on SSE-enabled kernels.
Matthew Brett
Initial work and help with getting ATLAS to build dynamic libraries.
Nicholas Coult
Initial version of AltiVec-enabled SGEMM.
Tony Castaldo
Discovered importance of issuing instructions in sets of 4 for PowerPC970, which allowed us to increase dgemm kernel performance from 75 to 82.5% of peak.
Markus Dittrich
Found shell technique allowing configure to pass multiple words as a single flag.
Dean Gaudet
Initial work on using CPUID in configure, Efficeon tuning information, and many informative atlas-devel discussions on related matters.
Kazushige Goto
His ev5/ev6 GEMM was used directly by ATLAS 3.6 and older if the installer answers "yes" to its use during the configuration procedure on an alpha processor. This results in a significant speedup over ATLAS's own GEMM codes, and is the fastest ev5/ev6 implementation we are aware of. Explicit support for alphas was dropped in ATLAS 3.8.0.
Jeff Horner
Initial versions of Level 3 BLAS tester/timer, C interface to the Level 3 BLAS, and non-generated complex matmul code.
Camm Maguire
SSE and SSE2 kernels enabling large speedups for Intel architectures. Also maintains ATLAS (along with other packages) for debian distribution.
Tim Mattox and Hank Deitz
Provided an extremely efficient SGEMM 3DNow! kernel for Athlon.
Viet Nguyen and Peter Strazdins
UltraSparc-optimized [D,Z]GEMM kernels.
Julian Ruhe
Incredibly efficient Athlon matmul kernel.
Peter Soendergaard
SSE and 3DNow! kernel work, enabling large speedups on Intel and AMD archs. Also, translation of Julian's Athlon kernels from NASM to gnu assembler, and extention to all precisions.
Carl Staelin
Initial work on parallelizing ATLAS make.

### ATLAS testers

Getting a stable release of ATLAS out the door is a seemingly never-ending task. One of the biggest headaches involves testing ATLAS, which can run on almost any platform, on enough architectures to be confident that it is sound enough to be called stable. We have testers that automate this process, but they typically hog up the entire CPU for more than a day, and no one likes to have to hassle with running them.

For ATLAS 3.8.0, I did all testing myself, though several developers provided me with machine access.

It should be completely safe to call any ATLAS routine from a threaded code. There are no global variables, or other shared information between routines. Probably the best idea is to say "yes" to threading in config, even if you wish to do the threading yourself. That way, the ATLAS lib will be compiled with the threading flags. Then, simply link to the serial interface so that ATLAS doesn't do the threading. If you want ATLAS to do the threading as well, simply link to the threaded interface.

### Can I vary the number of threads ATLAS uses dynamically?

No. The maximum number of threads to use is determined at compile time. ATLAS will never use more than this, but may use less if the problem sizes are too small to get speedup from the additional parallelism.

### What's the deal with the RHS in the row-major factorization/solves?

Most users are confused by the row major factorization and related solves. The right-hand side vectors are probably the biggest source of confusion. The RHS array does not represent a matrix in the mathematical sense, it is instead a pasting together of the various RHS into one array for calling convenience. As such, RHS vectors are always stored contiguously, regardless of the row/col major that is chosen. This means that ldb/ldx is always independent of NRHS, and dependant on N, regardless of the row/col major setting.

### Why don't you speedup/improve ATLAS's search

There are several questions here, handled in their own sub-questions: As you will see by reading each, only the last of these actually would be helpful for ATLAS's main use, and it has stayed on the backburner for quite some time because its almost always more useful to expand ATLAS's other capabilities. Note that the majority of users should use the provided architectural defaults, thus avoiding the search altogether. The search is there only for exploration by the expert user (in a user-controlled fashion), or to enable a naive user to get an adequate library on a truely new architecture (in its fully automatic mode).

### Why don't you improve the type of ATLAS's search?

There has been quite a bit of research on fast search techniques. ATLAS uses a relaxed 1-D line search, where the relaxed' comes from the fact that interacting transforms are usually handled by restricted 2/3-D searches. This is a very basic search technique, and many people wonder why a more advanced algorithm, such as hill climbing, simulated annealing, or genetic algorithm isn't used. The real answer is that it is overkill. Because I understand the transformations ATLAS attempts, and how they interact, I am able to target the relaxed line search appropriately. More advanced techniques are more appropriate when you know do not understand good start values for transforms and less about the interactions between optimizations and how to resolve them. The modified line search has some nice properties: it is easily guided by hand by the expert user in order to expore spaces more fully, and it is easy to understand and maintain.

### Why don't you improve the speed of ATLAS's search?

I occasionally get suggestions on how to speedup ATLAS empirical search. I know of a multitude of ways that I could do this. In my view, however, they are not worth the effort/risk at the present time. Most users should use the architectural defaults, skipping the search altogether. The only speed criteria that went into the search design was that it needed to be tolerable. The main purpose of ATLAS is to provide an optimized library, and once the search could produce that in a period of time O(1 day), that seems good enough. Many architectures are much faster than that, of course.

There are many places in the search where I could prune things back and have no effect on performance on any known architecture, but since the speed is adequate, additional search options are left on in case an unexpected architectural change is found. I could also utilize more sophisticated sampling techniques, but these would then need to be validated to work on the vast array of machines (the present search having been tested for over seven years, and on innumerable architectures). All this is to say that speeding up the search is not a bad thing, it just is not that helpful to the core usage of ATLAS, and so it is not worth the cost/risk of change at this time. If additional tuning capabilities are added, so that the search time becomes more critical, then of course the search will be updated.

### Why don't you improve the accuracy of ATLAS's search?

This is the search problem that I am most tempted to fix. The present search is mainly designed to be usable by an installer with no system priveledges, who must install on stock systems that are experiencing unrelated load during the installation. Thus, by default ATLAS uses CPU-time for all non-threaded installation decisions, which is extremely innaccurate. This often leads to the search going awry (i.e., failing to find a more optimal kernel), which is why the architectural defaults are so important.

For most systems, you can at least tell ATLAS's configure to use the cycle-accurate wall timers, which will make all timings much more accurate if you are on an unloaded machine.

Ultimately, a search with some real statistics should be built, which would determine if two timings are statistically different or not, and also use some statistics to see how many probes are required to get reliable results. This is the area that I think I a most likely to improve the search in, if I ever have time.

### What's the deal with the architectural defaults?

I split this into several seperate questions:

### Is using the architectural defaults important, rather than doing my own search?

The short answer is definitely. As described elsewhere the search is designed to be used only when architectural defaults are unavailable or have become non-optimal due to compiler change. To understand this, you need to understand the nature of empirical searches in general.

Empirical searches, when ran on real machines experiencing unrelated load, are almost never strictly repeatable, even in the best of cases. The default ATLAS search is far from the best case: the sampling and timing mechanisms are crude, made to work on the lowest-common denominator setups, up to and included embedded systems. So, when run in this mode, the search is designed to give you a library that isn't bad, but is often far from the best.

To get better results (which are then saved as architectural defaults), I usually run the search multiple times, and if necessary, intervene by hand to probe promising transformations. Thus, the architectural defaults can be thought of as a save of several installs + some user intervention. Also, the architecural defaults are synergistic with the default compiler flags, so you want to leave both alone for best results.

### When should I not use architectural defaults?

As previously mentioned, architectural defaults are usually the result of several guided installations, and thus represent best of breed installs. They can become a barrier to performance occasionally, particularly when a compiler goes through a major release. For instance, ATLAS 3.8.0's architectural defaults are for gcc 4.2, and you are presently using 5.1, it might be possible that things have changed enough to require new defaults, and if you are using a bad compiler like gcc 4.1 or an old one like gcc 3, you will almost certainly not want to use the architectural defaults.

The first thing to check is that your library runs about the same as those that you would get using the architectural defaults. You can determine this by make time, as described here.

If you choose not to use the recommended compiler and architectural defaults, be sure to follow these directions. If you suspect your performance is suboptimal, open up a support request and ask.

### If I don't use architectural defaults, how can I get better performance?

First, make sure your defaults are better than the architectural defaults by comparing the timings of a default install against your search install, as described here.

Play with the different compiler and flags to find things that better match both the defaults, and your output flags. Be sure to do all the normal post-install tuning, including tuning CacheEdge. Finally, if your install is indeed faster than the arch defaults, report it.

### Why does the search limit NB to 80?

The default ATLAS search limits GEMM's blocking factor to at most 80. On systems where larger NB actually blocks for the L2, blocking for the L2 prevents ATLAS from using it's multilevel blocking parameter, CacheEdge. In this case, larger blockings may result in superior kernel timings (which do no L2 blocking), but if an L1-contained NB is used, similar or superior performance may be obtained in full GEMM with a tuned CacheEdge. In this case, the GEMM speedup is illusory, but the application and small-case gemm slowdown (discussed below) is quite real. On machines with large L1, or very fast L2, GEMM may indeed get a asymptotic speedup from larger blocking factors, but it is still almost always a bad idea, as outlined below.

There are three main factors why even true asymptotic speedup from large blocking factors are a bad idea:

1. Overall GEMM performance is usually decreased, because more time is spent in cleanup (called when one or more dimensions are not a multiple of NB). For many GEMM calls, very large NBs result in calling nothing but the relatively poorly optimized cleanup.
2. Application performance, which usually calls GEMM with varying sizes, most of which are of modest dimension, spend even greater proportion of their time in cleanup code, and thus experience much greater slowdown than individual GEMM calls.
3. Many applications have unblocked code, and in some cases, the blocking factor may not necessarily match GEMMs. In this case, a large NB can guarantee that all GEMM calls go straight to cleanup. Even when this doesn't happen, as NB rises, the number of calls to GEMM goes down, and since large NB gemm requires larger matrices to reach its asymptotic peak, even recursive factorizations (the best case for large-NB GEMM) need to get extremely high efficiency for the large GEMM cases to overcome the slowdown on the small cases. For fixed-NB cases as in the LAPACK factorizations, large NB are almost always a loss. This is the main reason that ATLAS is reluctant to use large NB.

Note that points (2) & (3) are very important: GEMM is one of the most studied performance kernels in the world not for its own sake, but due to the wide variety of applications whose performance can be improved by speeding it up. Thus, speeding up GEMM at the expense of application performance is something that only someone interested in benchmarking GEMM (as opposed to building a usuable library) would want to do.

Therefore, the ATLAS search limits NB to 80. We occasionally relax this limit (manually, never blindly in the search) when it is absolutely necessary. For instance, on SPARCs, large NB have proven necessary for decent performance, and on the Pentium 4 (not P4E), the floating point unit does not make use of the L1 cache, and so we block for the L2. However, in these cases we first verified that the win is true and substantial, and we then hand-tuned the cleanup to ameloriate the effects of large NB as best we could. Even so, these systems can display very bad performance due to point (3) above, and we actually do not use the best NB for GEMM performance even so, as we increase it only large enough to get adequate asymptotic performance. Without examining this tradeoffs, you should never increase NB, unless you are tuning for a large GEMM benchmark.

### Does ATLAS provide a standard C interface to LAPACK?

The short answer is no, and neither does anyone else. As far as I know, there has been no official standardization of a C interface to LAPACK. In the absence of a standard, each library is free to do things differently. For instance, netlib provides something called clapack, which is the result of running lapack through f2c on a particular platform. This means all paremeters must be passed by reference, names have an underscore appended, etc. ATLAS does not support this adhoc interface, though you can use ATLAS to provide the BLAS for netlib clapack, as mentioned here.

I believe most of the vendors provide some C interface to LAPACK; I think most of them just use the same name (eg, F77's DGESV becomes dgesv), and the scalars become pass-by-value. ATLAS provides a C interface to the LAPACK routines natively provided by ATLAS, which is based on the standardized C interface to the BLAS. These routines are prefixed with clapack_, and their prototypes can be found in ATLAS/include/clapack.h. ATLAS natively provides only a handful of LAPACK routines, so if you want to call something that is not provided here, your best bet is to call the Fortran77 interface. The good news about this is that being a standard interface, all lapack libraries should support it in the same way.

### Why are you such a jerk when answering user questions? AKA: how can I help you feel good about providing me with support?

It is one of the unfortunate realities of open source development that one of the few rewards that it should supply turns out in practice to be a string of disparagement. I am talking, of course, about corresponding with the people who are using the software you have produced and supported, free of charge to them.

I understand why. A user has a problem using/installing/understanding the software, and is understandably frustrated. Only when the frustration has built up to a great degree is he/she motivated to write to the author. At that point, the user emotionally feels that the author has done it to him on purpose, and so, usually without realizing it, attacks the author.

What users should consider is that for an open source developer, support requests constitute 90% of his contact with his users. If all of this contact is negative, it does not lead to a desire on the author's part to provide a lot more support, and in extreme cases, probably causes developers to quit the project.

My guess is I get probably 2 or 3 message per year that have something positive to say about the software that I spend enormous amounts of my life developing, maintaining and supporting, and provide to the user for free. For every positive message, I get many many insulting or denigrating replies. It always adds to the anger quotient to feel that the user thinks so little of you that he thinks nothing of insulting you while using your software and asking for your help, even though you don't know him, he's not intending to do anything for you, and you are providing this stuff for free.

So, I have added this discourse to the FAQ in the hopes of stimulating users to consider the tone of their messages to me, but also to open source projects in general. Remember that the open source developer is not a vendor such as Microsoft or AOL, or even Red Hat, who is in a financial relationship with you, and thus owes you a lot of hand-holding. Instead the author has given all users of the package a gift, and you are now asking for additional help in using it.

Another thing to keep in mind when you are writing your mail is that it is entirely possible the question you are asking is answered in the documentation. If I get several questions about the same topic, I usually wind up creating a FAQ or errata entry about it. In a perfect world, this would mean you would read the docs and not have to ask it. Even assidiously trying to do so, it is easy to miss the relavant doc. So, it is a good idea to couch your language so that the author is not tempted to reply: RTFM, @ASD@!. If a user sends in a mail Hey, I can't get these files to link, any idea what's wrong?, I don't mind giving him the link to the errata entry that's been there since version 0.0.1 of the software. However, the guy with the more common Your libraries do not work style of message simultaneously demotivates me for work on the project, pisses me off, and generates a great desire to return the anger to him with a good old RTFM diatribe.

Even if you are not going to read through all the docs, if you are submitting a support request, at least take the time to read the FAQ entry on how to submit a support request. Almost half my users send in a message that translates to I haven't read the docs, and they post it to the "bugs" list, which is reserved for developer-verified bugs.

Let me show the absolute best kind of support request I get:

Subject: Problems with 3.5.8
Hi,

I've been using it in my chemistry research in order to do XXX for a couple
of years now, and ATLAS is a great piece of software!  However, I'm
now having a problem getting the newest release to work.

I'm already aching to help this guy. Not only has he indicated he appreciates what I've produced, he's given me an idea of what he is using it for (something I am always interested in). He's also not prejudged that the software or I am wrong. He's having a problem, which he later describes in detail, including the error report. If it's a bug, I'll tell him so and post a fix. It its a user error, I won't mind letting him know the fix, and will feel more like helping him again sometime.

As I said, I get maybe one or two messages of this type a year. I get quite a few absolutely neutral messages, which are OK as well. They don't imply that the problem is necessarily in my software or my brain, but rather just report that a problem has been encountered. They go more like:

Subject : matvec problems
Whenever I call matvec with N=200, my install seems to get worse performance
than the reference BLAS.  Any idea why?  I include my timer below.

Thanks,
My Name

This is a simple request for help, that does allow for the interpretation that there might be an problem with the installation, or perhaps a timer error on the user's part, as well as the idea that ATLAS is screwed up. This is good, because the majority of user requests turn out not to be errors in ATLAS. Nonetheless, here is a more typical phrasing:
Subject : error in matvec
Your matrix-vector product has an error.  For N=200, it is slower than the
`


text:
## » On NixOS, GPU programming and other assorted topics

June 23, 2018 | cpp development english linux | Adrian Kummerländer

So I recently acquired a reasonably priced second-hand CAD workstation computer featuring a Xeon CPU, plenty of RAM as well as a nice Nvidia K2200 GPU with 4 GiB of memory and 640 cores as the heart of the matter. The plan was that this would enable me to realize my long hedged plans of diving into GPU programming - specifically using compute shaders to implement mathematical simulation type stuff. True to my previously described inclination to procrastinate interesting projects by delving into other interesting topics my first step to realizing this plan was of course acquainting myself with a new Linux distribution: NixOS.

So after weeks of configuring I am now in the position of working inside a fully reproducible environment declaratively described by a set of version controlled textfiles1. The main benefit of this is that my project-specific development environments are now easily portable and consistent between all my machines: Spending the morning working on something using the workstation and continuing said work on the laptop between lectures in the afternoon is as easy as syncing the Nix environments. This is in turn easily achieved by including the corresponding shell.nix files in the project’s repository.

As an example this is the environment I use to generate this very website, declaratively described in the Nix language:

with import <nixpkgs> {};

stdenv.mkDerivation rec {
name = "blog-env";
env = buildEnv { name = name; paths = buildInputs; };

buildInputs = let
generate  = pkgs.callPackage ./pkgs/generate.nix {};
preview   = pkgs.callPackage ./pkgs/preview.nix {};
katex     = pkgs.callPackage ./pkgs/KaTeX.nix {};
in [
generate
preview
pandoc
highlight
katex
];
}

Using this shell.nix file the blog can be generated using my mostly custom XSLT-based setup2 by issuing a simple nix-shell --command "generate" in the repository root. All dependencies - be it pandoc for markup transformation, a custom KaTeX wrapper for server-side math expression typesetting or my very own InputXSLT - will be fetched and compiled as necessary by Nix.

{ stdenv, fetchFromGitHub, cmake, boost, xalanc, xercesc, discount }:

stdenv.mkDerivation rec {
name = "InputXSLT";

src = fetchFromGitHub {
owner = "KnairdA";
repo = "InputXSLT";
rev = "master";
sha256 = "1j9fld3sh1jyscnsx6ab9jn5x6q67rjh9p3bgsh5na1qrs40dql0";
};

buildInputs = [ cmake boost xalanc xercesc discount ];

meta = with stdenv.lib; {
description = "InputXSLT";
homepage    = https://github.com/KnairdA/InputXSLT/;
};
}

This will work on any system where the Nix package manager is installed without any further manual intervention by the user. So where in the past I had to manually guarantee that all dependencies are available which included compiling and installing my custom site generator stack I can now simply clone the repository and generate the website in a single command3.

It can not be overstated how powerful the system management paradigm implemented by Nix and NixOS is. On NixOS I am finally free to try out anything I desire without fear of polluting my system and creating an unmaintainable mess as everything can be isolated and garbage collected when I don’t need it anymore. Sure it is some additional effort to maintain Nix environments and write a custom derivation here and there for software that is not yet available4 in nixpkgs but when your program works or your project compiles you can be sure that it does so because the system is configured correctly and all dependencies are accounted for - nothing works by accident5.

Note that the nix-shell based example presented above is only a small subset of what NixOS offers. Besides shell environments the whole system configuration consisting of systemd services, the networking setup, my user GUI environment and so on is also configured in the Nix language. i.e. the whole system from top to bottom is declaratively described in a consistent fashion.

NixOS is the first destribution I am truly excited for since my initial stint of distro-hopping when I first got into Linux a decade ago. Its declarative package manager and configuration model is true innovation and one of those rare things where you already know that you will never go back to the old way of doing things after barely catching a climpse of it. Sure, other distros can be nice and I greatly enjoyed my nights of compiling Gentoo as well as years spent tinkering with my ArchLinux systems but NixOS offers something truely distinct and incredibly useful. At first I thought about using the Nix and Scheme based GuixSD distribution instead but I got used to the Nix language quickly and do not think that the switch to Guile Scheme as the configuration language adds enough to offset having to deal with GNU’s free software fundamentalism6.

Of course I was not satisfied merely porting my workflows onto a new superior distribution but also had to switch from i3 to XMonad in the same breath. By streamlining my tiling window setup on top of this Haskell-based window manager my setup has reached a new level of minimalsim. Layouts are now restricted to either fullscreen, tabbed or simple side by side tiling and everything is controlled using Rofi instances and keybindings. My constant need of checking battery level, fan speed and system performance was fixed by removing all bars and showing only minimally styled windows. And due to the reproducibility7 of NixOS the interested reader can check out the full system herself if he so desires ! :-) See the home-manager based user environment or specifically the XMonad config for further details.

After getting settled in this new working environment I finally was out of distractions and moved on to my original wish of familiarizing myself with delegating non-graphical work to the GPU. The first presentable result of this undertaking is my minimalistic fieldplay clone computicle.

What computicle does is simulate many particles moving according to a vector field described by a function $f : \mathbb{R}^2 \to \mathbb{R}^2$ that is interpreted as a ordinary differential equation to be solved using classical Runge-Kutta methods. As this problem translates into many similiar calculations performed per particle without any communication to other particles it is an ideal candidate for massive parallelization using GLSL compute shaders on the GPU.

#version 430

layout (local_size_x = 1) in;
layout (std430, binding=1) buffer bufferA{ float data[]; };

vec2 f(vec2 v) {
return vec2(
cos(v.x*sin(v.y)),
sin(v.x-v.y)
);
}

vec2 classicalRungeKutta(float h, vec2 v) {
const vec2 k1 = f(v);
const vec2 k2 = f(v + h/2. * k1);
const vec2 k3 = f(v + h/2. * k2);
const vec2 k4 = f(v + h    * k3);

return v + h * (1./6.*k1 + 1./3.*k2 + 1./3.*k3 + 1./6.*k4);
}

[...]

void main() {
const uint i = 3*gl_GlobalInvocationID.x;
const vec2 v = vec2(data[i+0], data[i+1]);
const vec2 w = classicalRungeKutta(0.01, v);

data[i+0]  = w.x;  // particle x position
data[i+1]  = w.y;  // particle y position
data[i+2] += 0.01; // particle age
}

As illustrated by this simplified extract of computicle’s compute shader, writing code for the GPU can look and feel quite similar to to targeting the CPU in the C language. It fits that my main gripes during development were not with the GPU code itself but rather with the surrounding C++ code required to pass the data back an forth and talk to the OpenGL state machine in a sensible manner.

The first issue was how to include GLSL shader source into my C++ application. While the way OpenGL accepts shaders as raw strings and compiles them for the GPU on the fly is not without benefits (e.g. switching shaders generated at runtime is trivial) it can quickly turn ugly and doesn’t feel well integrated into the overall language. Reading shader source from text files at runtime was not the way I wanted to go as this would feel even more clunky and unstable. What I settled on until the committee comes through with something like std::embed is to include the shader source as multi-line string literals stored in static constants placed in separate headers. This works for now and offers at least syntax highlighting in terms of editor support.

What would be really nice is if the shaders could be generated from a domain-specific language and statically verified at compile time. Such a solution could also offer unified tools for handling uniform variables and data buffer bindings. While something like that doesn’t seem to be available for C++8 I stumbled upon the very interesting LambdaCube 3D and varjo projects. The former promises to become a Haskell-like purely functional language for GPU programming and the latter is an interesting GLSL generating framework for LISP.

I also could not find any nice and reasonably lightweight library for interfacing with the OpenGL API in a modern fashion. I ended up creating my own scope-guard type wrappers around the OpenGL functionality required by computicle but while what I ended up with looks nice it is probably of limited portability to other applications.

// simplified extract of computicle's update loop
window.render([&]() {
[...]
if ( timer::millisecondsSince(last_frame) >= 1000/max_ups ) {

last_frame = timer::now();
}
[...]
{
auto texGuard = texture_framebuffers[0]->use();

[...]
particle_buffer->draw();
}

{

glClear(GL_COLOR_BUFFER_BIT);

display_buffer->draw(textures);
}
});

One idea that I am currently toying with in respect to my future GPU-based projects is to abandon C++ as the host language and instead use a more flexible9 language such as Scheme or Haskell for generating the shader code and communicating with the GPU. This could work out well as the performance of CPU code doesn’t matter as much when the bulk of the work is performed by shaders. At least this is the impression I got from my field visualization experiment - the CPU load was minimal independent of how many kiloparticles were simulated.

1. See nixos_system and nixos_home

2. And this works on all my systems, including my Surface 4 tablet where I installed Nix on top of Debian running in WSL

3. Which is not a big problem in practice as the repository already provides a vast set of software and builders for many common build systems and adapters for language specific package managers. For example my Vim configuration including plugin management is also handled by Nix. The clunky custom texlive installation I maintained on my ArchLinux system was replaced by nice, self-contained shell environments that only provide the $\LaTeX$ packages that are actually needed for the document at hand.

4. At least if you are careful about what is installed imperatively using nix-env or if you use the pure flag in nix-shell

5. Which I admire greatly - but I also want to use the full power of my GPU and run proprietary software when necessary

6. And the system really is fully reproducible: I now tested this two times, once when moving the experimental setup onto a new SSD and once when installing the workstation config on my laptop. Each time I was up and running with the full configuration as I left it in under half an hour. Where before NixOS a full system failure would have incurred days of restoring backups, reconstructing my specific configuration and reinstalling software I can now be confident that I can be up and running on a replacement machine simply by cloning a couple of repositories and restoring a home directory backup.

7. At least when one wants to work with compute shaders - I am sure there are solutions in this direction for handling graphic shaders for gaming and CAD type stuff.

8. Flexible as in better support for domain-specific languages

## » Notes on BlackBerry OS 10 development in 2017

October 3, 2017 | cpp development english | Adrian Kummerländer

I recently broke my seven-year streak of only using smartphones in the tradition of Nokia’s fabled N900. The reason for this change was the growing age of my Jolla phone combined with my continued desire for a working physical keyboard1. In today’s market overflowing with faceless, exchangeable, uninspired and uniform touch-only devices this left me no choice but to opt for the last true BlackBerry: The Passport.

### Thoughts on OS 10

The BlackBerry Passport is the last device that was both manufactured by RIM and runs their QNX based OS 10 operating system. OS 10 offers everything you can expect from a state of the art smartphone operating system. Furthermore its communication stack is beyond comparison and renders any third-party mail, calendar, task and contact applications redundant - as one would expect of a device built by the erstwhile mobile handset pioneer. The system integrates perfectly with the hardware and features true multitasking that is exposed to the user in a Maemo-like fashion. One very nice feature is the keyboard’s touch capability that allows for text-navigation and scroll movements without touching and thus obscuring the actual screen. Although the native app catalog obviously doesn’t feature as many applications as are available on the big platforms I don’t miss anything2.

QNX is a unixoid real-time operating system that is very interesting in its own right. A nice side benefit of this is the ability to run a native terminal emulator and execute cross-compiled UNIX tools in the way I got used to during my time with Maemo and Sailfish OS. Sadly BlackBerry OS 10 is deprecated and won’t receive future support as the whole platform has been discontinued3.

Luckily I found the system to be very stable and feature complete. Thus I don’t see any immediate problems requiring continued support - as long as I can compile and install my own code.

OS 10 application development is based mostly on C++ and Qt/QML which suits my preferences quite nicely. Additionally the default stack includes a Python 3.2 interpreter as well as the ability to install and run arbitrary APKs.

### Momentics IDE

BlackBerry offers a native SDK which bundles a Eclipse-based IDE called Momentics in addition to a full set of CLI utilities spanning the full development process. Installing and running it on my preferred distribution was straight forward. The only issue I encountered was the need to manually pass the path to an older XUL runner instance:

./qde -Dorg.eclipse.swt.browser.XULRunnerPath=~/tmp/xul/xulrunner/

~/tmp/xul/xulrunner contains the extracted contents of a matching XUL runtime version 10.0 archive.

While enabling the development mode in the device’s settings menu is just a toggle flick away and only requires a device password to be defined actually installing a program archive generated by Momentics requires one to register with RIM in order to be able to sign the archive. Fortunately enough this process is completely free and devoid of any further organisatory hurdles.

When I first tried to connect to my Passport via the IDE I encountered an SSL authentication error. All my attempts at solving this by temporarily disabling authentication via JVM flags failed. This issue also extended to browser based device access using e.g. BB10 App Manager which failed due to a invalid server response. In fact the only thing that worked was the SSH connection initiated via the SDK’s CLI utilities:

./blackberry-connect $device_ip -password$device_password -sshPublicKey ~/.
rim/bbt_id_rsa.pub

After some investigation I was able to trace this issue to some custom system update trickery I performed during the first days of setting up the device4. Luckily this was easily solved by resetting the device to the latest OS 10 version via the official autoloader.

After solving this issue Momentics turned out to be a workable IDE that seems to be enough for most purposes. Should this turn out to be a premature assessment it will be worthwhile to check out older versions of Qt Creator which include BlackBerry 10 support. And if everything visual fails there are still the SDK’s CLI utilities that allow for fully text driven packaging, signing and deployment of applications:

λ ~/t/m/b/h/l/x/u/bin ● ls | grep blackberry
blackberry-airpackager
blackberry-barchecker
blackberry-connect
blackberry-custompackager
blackberry-debugtokenrequest
blackberry-deploy
blackberry-keytool
blackberry-nativepackager
blackberry-signer
blackberry-uripackager

### Device-local development

As the alert reader might have noticed everything described in the last section hinges on the continued willingness of BlackBerry to generate and distribute development certificates without which one cannot deploy applications to physical devices. This is probably one of the primary tradeoffs one makes when leaving the world of open mobile systems behind.

Luckily there is a way to cross compile, install and run arbitrary CLI programs without even enabling the development mode: mordak/playbook-dev-tools.

This set of scripts developed by the same person that wrote the premier native terminal emulator for OS 10 offers a fully automated systems that compiles and installs GCC 4.6.3 amongst further development and system utilities.

It remains to be investigated if this entry point can be expanded to device-local development of full Cascades-based UI applications.

1. i.e. something better than the TOHKBDv2

2. Especially since it offers an Android runtime analogously to what I am used to on Sailfish OS

3. I am not convinced by RIM’s externally manufactured Android smartphone. Neither the build quality nor the software are anywhere near to what they offered with the Passport. I fear that they will further degrade the keyboard quality in future devices and expect them to fully disappear from the market within the next couple of years.

4. Namely impatiently forcing an OS upgrade using Sachesi

## » Tinkering with meta tools

January 20, 2017 | english linux opinion | Adrian Kummerländer

If the contents of this website are of interest to you chances are that you also know how easy it is to spend large parts of one’s leisure time tinkering with computer systems and improving workflows while losing sight of what one actually wants to accomplish. I am going to use this article to document parts of my current personal computing setup in an attempt to summarize and refocus my efforts in this direction in addition to articulating some of my thoughts in this context.

Working with computers is my job to the extend that computers are the machines that my software developing self instructs to - hopefully - solve actual problems in the real world™. This means that configuring and optimizing my personal computing environments is not only a leisure time activity but can be considered a productive activity within certain boundaries1. If I think of time consuming optimization-activties that are definitely yielding payoff in my day to day work a few things come to mind: Learning and mastering a single powerful text editor2, switching to an open operating system and using open source software wherever possible, using a tiling window manager, version control everything, maintaining a private internet archive, frequently experimenting with new programming languages and paradigms as well as studying mathematics. I am going to use the following sections to further explore some of these activities.

### Text editing

Plain text is still the most used format for expressing and storing program instructions for all common and nearly all uncommon languages out there. Although the simplicity and thus flexibility of plain text continues to cause distracting discussions around topics such as tabs-vs-spaces3 there sadly are no usable alternative in the vein of e.g. generalized tree encodings available at this point in time. S-Expressions could probably be used as a foundation for such an alternative but alas we don’t live in the alternative time line where the adoption of a LISP as the internet’s scripting language instead of the quickly hacked together thing that is JavaScript caused a widespread rise of LISP-like languages culminating in all common languages being LISP-dialects and thus being expressible in sexprs.

To return the topic at hand: plain text it is and will continue to be for the foreseeable future. If one looks for powerful, widely available and extensible text editors the choice will in most cases come down to the still reigning giants Vim and Emacs 4.
When I stood before this decision nearly half a lifetime ago5 my choice fell on Vim. If one only considers the default editing interface this definitely was the right choice for me as I would not trade Vim’s modal text manipulation language for overly long default Emacs key chords. But I have to admit that I regularly take wistful looks at some of the applications available inside Emacs such as Org-mode. Had I already discovered the joys of LISP when I was trying to decide between the two editors I probably would have chosen Emacs. Luckily one can always alter such decisions - sooner or later I will probably settle on some kind of Evil based setup just to be able to use the text based operating system that is Emacs.

Altough the 122 lines including comments that currently make up most of my Vim configuration are not much by most standards I have invested quite some time in setting the editor up to my taste - not the least by writing my very own color scheme to match the rest of my primary desktop environment.

Over time I have grown so accustomed to Vim’s keybindings that I now use them wherever possible including but not restricted to: my web browser, PDF reader, various IDEs, image viewer and window manager of choice.

### Operating system and desktop environment

ArchLinux continues to be the Linux distribution of choice for most of my computing devices. It offers most of the flexibility of Gentoo if one needs it while preserving fast installation and frequent updates of most of the software I require. The only feature I currently miss and that will probably lead me switch distributions sooner or later is declarative package management as offered by NixOS or GuixSD. A fully declarative configuration of my Linux installation in a plain-text and thus easily version controllable file would be the logical conclusion of my current efforts in managing system configurations: /etc is tracked using etckeeper, various dotfiles are tracked using plain git and GNU stow for symlink management.

My choice of desktop environment has converged on a custom setup built around the i3 tiling window manager over the last couple of years. I have adopted the tiling concept to such a degree that I struggle to think of a single advantage a common - purely floating - window manager might hold over tiling window managers. This holds especially true if your workflow is similar to mine, i.e. you have lots and lots of terminals and a few full screen applications running at most times as well as a fondness for using the keyboard instead of shoving around a mouse.

What I like about i3 in particular is that it doesn’t enforce a set of layouts to toggle between like some other tiling WMs but allows you full control over the tree structure representing its layout. Another nice feature is that it lends itself to Vim-style keybindings as well as offering good support for multi monitor setups.

Compared to a desktop environment such as KDE a custom setup built around a window manager obviously requires much more configuration. In exchange for that I gained full control over my workflow in addition to basically instantaneous interaction with the system and its applications.

I find it very useful to have a certain set of information available at all times. Examples for such information are dictionary definitions, language and framework documentation, login credentials and notes. For this purpose I wrote a set of scripts that enable me to query a local dict daemon6, note folders and passman entries using the dmenu-replacement Rofi. In addition to that i3’s scratchpad feature is very useful for keeping Zeal and Vim instances running in the background while preserving accessability in every context by displaying them in a floating window overlaying the current workspace.

### Archiving, web browsing and note taking

An important purpose of the tool we call Computer is accessing information on the Internet. This task is achieved primarily using a web browser - in fact there is an argument to be made that web browsers are some of the most complex applications the average users comes into contact with.
I sometimes frown at the wasteful complexity many of today’s websites contain even if their actual contents still consist of mostly plaintext and some pictures. It is no longer the exception but the rule that a single load of common websites pulls more data over the wire than would be required to encode a whole book while often containing far less content. These are the moments where I wish that Gopher had gained wider usage or that something in the vein of ipfs will grow to encompass most of the sources I commonly use7.

As one can see in the screeshot above the current browser setup I use to deal with this rather unsatisfying state of things is a quite customized version of Firefox. This is achieved using Pentadactyl, TreeStyleTabs and ScrapBook X.

Pentadactyl transforms Firefox into a fully keyboard driven browser with Vim-like keybindings and looks as well as the possibility of configuring the browser using a single dotfile .pentadactylrc.

TreeStyleTabs allows me to better manage my workflow of keeping all tabs related to my current projects and interests opened as well as grouped in a tree structure. While Vim-like keybindings can be configured in other browsers, usable TreeStyleTabs like tab management is currently only available on Firefox.

ScrapBook X is my current answer to a question I have spent quite some time thinking about: How to best archive and organize interesting documents on the web. ScrapBook allows reasonably good offline archival of websites, organizing archived pages in a folder structure and exporting a HTML version of itself for hosting a private Internet Archive. It doesn’t work perfectly on all websites and uses plain file mirroring instad of the more suitable WARC format but it is the best solution I was able to find short of investing most of my time into attemting to develop something more to my taste.
For now it is good enough and fullfills my primary use cases: Having access to most of my sources independent of network availability8, preventing the unpleasant situation of wanting to consult a vaguely remembered source only to find that it is not available anymore as well as full text search over all interesting pages I visited.

Archiving the web ties into the last section of this article: note taking. While I write all my lecture notes and excercise sheet solutions using pen input on one of Microsoft’s Surface9 devices I like to capture project and research notes as well as general thoughts using a keyboard on my normal computer.
When taking notes as plain text I preferably want to do so using Vim which rules out most of the already relatively limited selection of open source desktop wiki software. After quite some time using VimWiki I currently use markdown files stored in a flat directory structure. Desktop integration is solved using a background Vim instance running in a i3 scratchpad as well as a Rofi based note selection dialog that communicates with Vim using remote commands. Advanced markdown features, syntax highlighting and conversion is based on pandoc integration plugins.
In addition to that I also now and then play around with Emacs’ Org-mode which can probably fulfill most of my requirements but requires considerable upfront configuration efforts to make it work consistently with Evil for Vim-style usage.

### Vision

While the setup outlined above is workable I definitely do not consider it a permanent solution. Even more so I feel that there is much unexplored potential in augmenting not just note taking but formal and exploratory thinking in general using computers. What I mean by that are systems as they were envisioned in the form of the Memex or as described by Engelbart in Augmenting Human Intellect10.

Such a system would not only encompass note taking and knowledge management but form an intergral part of the user facing operating system. Everything you see on a screen should be explorable in the vein of Lisp Machines or Smalltalk instances, annotatable in a fully interlinkable and version controlled knowledge base that transparently integrates information from the Internet in a uniform fashion as well as shareable with other beings where required.
Formal verification software could look over the figurative shoulder of the user and point out possible fallacies and formal errors. Even a single component of such a system like for example a math environment that pulls up appropriate definitions, theorems and examples depending on the current context or a literate development environment that automatically pulls up documentation, stackexchange threads and debugging output would be of great help.
Usage tracking similar to Gnome Zeitgeist in conjunction with full text archival of all visited websites and fully reproducible tracking of the desktop state would also complement such a system.

In conclusion: My current setup doesn’t even scratch the surface of what should be possible. Tinkering with my computing systems and workflow-optimization will continue to be an unbounded but at least somewhat productive leisure time sink.

1. …that I have probably crossed frequently

2. Section 16, Power Editing, The Pragmatic Programmer.

3. Tabs for indentation, spaces for alignment it is for me - this is probably the reason for my dislike of whitespace-dependent languages that tend to interfere with this personal preference of mine.

4. Desktop-only text editors - especially those Electron based ones written in HTML and JavaScript of all things - are not an alternative for my use cases. Although at least VSCode is developing nicely into a usable cross platform IDE.

5. Luckily not that long in my case :-)

6. Serving a collection of freely available dictionaries such as Webster’s 1913 and Wordnet

7. Note to self: Make the raw markdown sources (?) of this blog available on ipfs

8. e.g. when riding the black forest train from Karlsruhe to my home village in the Hegau.

9. The only non-Linux device I regularly use. Pen input on tablet devices has reached a point of general viability (concerning power usage and form factor) in recent years. I am actually very satisfied with my Surface 4 and Windows 10 as both a note taking and tablet device. For tinkering and security reasons I still hope to one day be able to use an open operating system and open note taking software on such a device but for now it is a workable solution for my use case of studying (mostly) paperless.

10. AUGMENTING HUMAN INTELLECT : A Conceptual Framework., October 1962, D. C. Engelbart

format: plain


text:
# how to calculate cross correlation of transient signal?

there.

Let me have a noisy speech signal $y(k)$ as below.

$$y(k) = x(k) + v(k)$$

where, $k$ is the discrete-time index, $x(k)$ is clean speech signal, $v(k)$ is a zero-mean random process, uncorrelated with $x(k)$ e.g. white noise.

If we put into this as a vector form:

$$\textbf{y} (k) = \textbf{x}(k)+\textbf{v}(k)$$

where,

$$\textbf{y}(k) = [y(k) \text{ }\ y(k-1) \text{ }\ ... \text{ }\ y(k-L+1)]^T,$$

$L$ is a length of vector.

If we try to calculate the correlation matrix of size $L \times\ L$, the noisy speech signal can be written as

$$\textbf{R}_y = E[\textbf{y}(k)\textbf{y}^T(k)] = \textbf{R}_x+\textbf{R}_v.$$

Also, $\textbf{R}_x = E[\textbf{x}(k)\textbf{x}^T(k)]$, $\textbf{R}_v = E[\textbf{v}(k)\textbf{v}^T(k)]$.

However, the speech signal is not the stationary signal but the transient signal. This correlation matrix also can be used for calculating MVDR(Minimum Variance Distortionless Response) process.

Here is what I want to know.

Because $\textbf{y}(k)$ is not the stationary signal, I think there is a method for calculating the correlation matrix for transient signal.

How can we calculate the cross-correlation matrix for transient signal?

Above article is in the below. If there is any problem, I do delete this post. Thank you.

Jingdong Chen, Jacob Benesty, Yiteng Huang, and Tomas Gaensler(2011), "On single channel noise reduction in the time domain," Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, On page(s): 277 - 280.

Correct speech signal is non-stationary. Hence, you have to stop here as whole dsp is based on stationary assumption, as of now at least to a larger extent. But as we cannot sit idle, we go ahead with quasi-stationary assumption. Take a window (rect/hamming/hanning - these have trade-offs) of 20-30 msec (this is different for male/female/music and is taken as 3-4 pitch periods in speech signal, you can keep it fixed at 30 msec (also referred as narrowband speech processing to get an intuition). Using this window break the speech signal into frames by windowing and shifting the window and extracting a frame for every window. You now have a bag of window length vectors of the given speech signal. Find auto-corr of these for any further processing as within the window the signal is assumed to be stationary (! though not exactly but better than the whole signal).

• Thank you for your comment. Yeah, I tried to calculate the speech signal as transient signal. So I questioned concern this theme. I need to study about quasi-stationary assumption. :) Also, let me summary what you mentioned. If I have one 30 msec frame, $F$. If I have a speech signal which length is $100F$, I have 100 frames if there is no overlap, right? In quasi-stationary assumption, each frame(30 msec signal) is treated as stationary signal? – actlee Dec 27 '13 at 6:39
• Yes, the moment you take auto-correlation on a signal that inherently means you have assumed stationary assumption in that frame. If the answer help you do give 1 up or else 1 down, forum should make the good answers highlighted! – Neeks Dec 27 '13 at 6:58
• Yeah, thanks. I would like to give up though I could not because I dont have enough reputation. :) However, this answer really helps me. Thanks. – actlee Dec 27 '13 at 7:12


text:
# Interpretation of a probability problem: expected value.

I am having a few doubt on the interpretation of this problem that I have read on book about interviews questions.

Here the text:

A mythical city contains N=100,000 married couples but no children. Each family wishes to continue the male lane but they do not wish to overpopulate. So, each family has one baby per annum until the arrival of the first boy. Assume that all the children are equally like to be born male and female (and independent).
Let $p(n)$ be the percentage of children that are male at the end of the year n. How is this percentage expected to evolve through time?

This is the problem and the solution says that the percentage is expected to remain constant at a level $\frac{1}{2}$.

#### Solutions Collecting From Web of "Interpretation of a probability problem: expected value."

During a given year, each family either (1) has a unique child or (2) has no child. Those who previously had a boy decide to have no supplementary child, hence these are all in case (2), but maybe some others are in case (2) as well, for other reasons of their own, this does not matter.

What matters is that each family in case (1) has as much chances to have a boy than a girl. By the law of large numbers, if the number $M$ of families who do procreate during this given year is large and if each procreates independently on the others, $\frac12M+r_M$ boys are born and $\frac12M-r_M$ girls are born, where $r_M$ is random and $|r_M|\ll M$. The proportion of boys amongst the children born this year is $\frac{M/2+r_M}M=\frac12+\varepsilon_M$ with $\varepsilon_M=\frac{r_M}M$ hence $|\varepsilon_M|\ll1$. In other words, roughly one half of all the children born this year are boys.

Thus the hypothesis that the global population is large is important, but the details of the strategy (in the present case, Stop after one boy) are simply not relevant since every adapted strategy (in the sense that the decision on a given year only depends on what happened on the previous years) would yield the same result.

Edit (This is to answer a comment by the OP.)

The preceding paragraphs describe the almost sure behaviour in the limit of large initial populations. Turning to the behaviour in the mean for finite initial populations, note that the distribution of $r_M$ is symmetric, since $r_M$ is the sum of a random number $M$ of i.i.d. centered $\pm1/2$ Bernoulli random variables. Hence $\mathrm E\left(\frac{b_k}{g_k+b_k}\right)=\frac12$ exactly, where $b_k$ and $g_k$ denote the numbers of boys and girls born in generation $k$.

This does not imply that the total numbers $B_k=b_1+\cdots+b_k$ and $G_k=g_1+\cdots+g_k$ of boys and girls born until generation $k$ fulfill the same property.

Consider for example the second generation. Then the distribution of $b_1$ is binomial $(N,\frac12)$, $g_1=N-b_1$, the conditional distribution of $b_2$ conditionally on $b_1$ or $g_1$ is binomial $(g_1,\frac12)$, and $g_2=g_1-b_2$. In particular, $\mathrm E(b_1)=\frac12N$ and $\mathrm E(b_2\mid b_1)=\frac12(N-b_1)$.

Consider the successive ratios $R_k=\frac{B_k}{B_k+G_k}$. Then $R_1=\frac{b_1}N$ hence $\mathrm E(R_1)=\frac12$. On the other hand, $R_2=\frac{b_1+b_2}{N+g_1}=\frac{b_1+b_2}{2N-b_1}$ hence $\mathrm E(R_2\mid b_1)=\frac{b_1+(g_1/2)}{2N-b_1}=\frac12\frac{N+b_1}{2N-b_1}$. By convexity,
$$\mathrm E(R_2)\gt\frac12\frac{N+\mathrm E(b_1)}{2N-\mathrm E(b_1)}=\frac12\frac{N+(N/2)}{2N-(N/2)}=\frac12,$$
hence $\mathrm E(R_2)\ne\frac12$.

Second edit

Counting the children family by family instead of generation by generation, one sees readily that $R_k\to R_\infty$ almost surely when $k\to\infty$, where $R_\infty=\frac{N}{N+\sigma_N}$ and $\sigma_N$ the sum of $N$ i.i.d. geometric random variables $\tau_i$ of parameter $\frac12$, such that $\mathrm P(\tau_i=n)=2^{-n}$ for every $n\geqslant0$. Further computations then show that
$$\mathrm E(R_\infty)=N\int_0^1\frac{u^{N-1}}{(2-u)^N}\mathrm du=\frac12+\frac34\frac1N+o\left(\frac1N\right).$$
In particular, $\mathrm E(R_k)\ne\frac12$ for every $k$ large enough (and probably for every $k\geqslant2$).

Since the probability of having a child of either particular gender is 1/2 and is independent, the percentage must be expected to remain constant at 1/2 boys and 1/2 girls. (Anytime anyone has a child we expect it to be a boy with probability 1/2, when they stop having children does not matter.)

Maybe it is illuminating to consider the first couple of years:
At the end of the first year, we expect 1/2 of the couples to have had boys and 1/2 to have had girls, so we have 1/2 of the children are boys, and 1/2 are girls (or 50,000 boys and 50,000 girls using the N = 100,000 couples of the problem).

At the end of the second year, all of the couples with girls will have another child, 1/2 of them expected to be boys, and 1/2 expected to be girls. Hence, the overall population of children will still be 1/2 boys and 1/2 girls. (So, in particular with N = 100,000, we expect 50,000 of these to have already had a boy, and 50,000 to have another child. Of these 50,000 we expect 25,000 to have boys and 25,000 to have girls. Hence, the population of children at the end of the second year is 75,000 boys and 75,000 girls.)

This will continue until every couple has a boy… The child population from the previous year will be 1/2 boys and 1/2 girls, and the new children born in a particular year will be 1/2 boys and 1/2 girls, leaving the ratio of boys to girls unchanged.


text:
Archived

This topic is now archived and is closed to further replies.

This topic is 5985 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I read several articles on quadtrees and I''m implementing one myself now. I just can''t see in what order I construct the quadtree. 1. I check south and east vertices if they should be enabled. 2. and then ????? If both edge vertices are enabled should I treat the SE quadrant as a new quad square (new node) ? But then the middle vertex of this quadrant isn''t enabled. Or should I first check if the SE needs subdivision before treating it as an separate (new) quad square. Can somebody explain the general lines of the quadtree algorithm? Thanks, Edo

Share on other sites
Just take Octree algorithem ( there is a bunch of tutorials one the net ) and use only 2 dimensions

There are more worlds than the one that you hold in your hand...

Share on other sites
I just finished a working quadtree demo (based on NeHe tut 35) if you want it.

Share on other sites

What exactly are you using the quadtree for?

My powers of telepathy tell me that you are using a quadtree as an level of detail terrain algorithm. If so, then this Gamasutra article will probably give you lots of tips:

http://www.gamasutra.com/features/20000228/ulrich_pfv.htm

Cheers

Keef

-----------------------
glDisable(WORK);
glEnable(GL_CODING);

Share on other sites
blueEbola: I do

------------------------------------------------------------
Email
Website

"If you try and don''t succeed, destroy all evidence that you tried."
------------------------------------------------------------

Share on other sites

Keef, I read the article on gamedev, and although it is pretty detailed in certain areas, it fails to make me clear what the precise procedure is on subdividing quads.

Yes, I''m using the quadtree for terrain LOD. BlueEbola: do you somewhere have pseudo-code description of your demo (or some site with your demo).

Thanks for the replies.

Edo

Share on other sites
I can probably upload it to my site tomorrow (I don't want to log into geocities and get involved with there bad interface right now)

but basically for my quadtree I define the size of the tree (as in the dimensions of the heightmap) and it with one call it builds the whole tree (using recursive function calls). In the function I set the bounding box for the node and then I check to see if its width is at my specified leaf size; if so, make it a leaf, otherwise it becomes a node. When its a node, it has children, when its a leaf, it doesnt. Heres what the function looks like (pseudo code):

  #define LEAF_WIDTH 16void CreateNode(QuadtreeNode *quadnode,int xstart,int xwidth,int ystart,int yheight){ // Define the bounding box here (I'm not going to write // all the code for this sorry) :( quadnode->Leaf = FALSE; if (xwidth == LEAF_WIDTH) { // Define all the vertices inside this leaf quadnode->Child[0] = NULL; quadnode->Child[1] = NULL; quadnode->Child[2] = NULL; quadnode->Child[3] = NULL; quadnode->Leaf = TRUE; } else { quadnode->Child[0] = new quadnode; quadnode->Child[1] = new quadnode; quadnode->Child[2] = new quadnode; quadnode->Child[3] = new quadnode; CreateNode(quadnode->Child[0],xstart,xwidth/2,ystart,yheight/2); CreateNode(quadnode->Child[1],xstart+xwidth/2,xwidth/2,ystart,yheight/2); CreateNode(quadnode->Child[2],xstart+xwidth/2,xwidth/2,ystart+yheight/2,yheight/2); CreateNode(quadnode->Child[3],xstart,xwidth/2,ystart+yheight/2,ywidth/2); }}

Thats basically how you can construct the quadtree. The QuadtreeNode structure could look like:

  typedef struct _quadnode{ BOOL Leaf; Vertex3f BoundingBox[4]; Vertex3f Vertices[LEAF_WIDTH][LEAF_WIDTH]; _quadnode *Child[4];}

I don't store the actual vertices in my quadtree structure though, I store indices which point into an array. I hope this helps a bit

Edited by - blueEbola on February 28, 2002 9:25:48 PM

1. 1
Rutin
26
2. 2
JoeJ
20
3. 3
4. 4
5. 5

• 10
• 10
• 9
• 9
• 10
• Forum Statistics

• Total Topics
631751
• Total Posts
3002086
×


text:
# Introduction to Mathematical Physics/N body problem and matter description/Other matter arrangements

## Quasi crystals

Quasi crystals ([#References|references]) discovered by Israeli physicist Shechtman in 1982 correspond to a non periodical filling of a volume by atoms or molecules. Pentagon is a forbidden form in crystallography: a volume can not be filled with elementary bricks of fifth order symmetry repartited periodically. This problem has a two dimensional twin problem: a surface can not be covered only by pentagons. English mathematician Penrose \index{Penrose paving}discovered non periodical paving of the plane by lozenges of two types (see figure ---figpenrose--- and figpenrose2).

figpenrose

File:Penrose
Penrose paving: from two types of lozenges, non periodical paving of the plane can be constructed.

figpenrose2

File:Penrose2
Penrose paving: same figure as previous one, but with only two gray levels to distinguish between the two types of lozenges.

Obtained structures are amazingly complex and it is impossible to encounter some periodicity in the paving.

secristliquides

## Liquid crystals

Liquid crystals ([#References|references]), also called mesomorphic phases\index{mesomorphic phase}, are states intermediary between the cristaline perfect order and the liquid disorder. Molecules that compound liquid crystals\index{liquid crystal} have cigar--like shape. They arrange themselves in space in order to form a "fluid" state more or less ordered. Among the family of liquid crystals, several classes can be distinguished.

In the nematic \index{nematic} phase (see figure ---fignematique---), molecular axes stays parallel. There exists thus a privileged direction. Each molecule can move with respect to its neighbours, but in a fish ban way.

File:Nematique
Nematic material is similar to a fish ban: molecules have a privileged orientation and located at random points.}
fignematique

When molecules of a liquid crystal are not superposable to their image in a mirror, a torsion of nematic structure appears: phase is then called cholesteric \index{cholesteric} (see figure ---figcholesteric---).

figcholesteric

File:Cholesteric
In a cholesteric phase, molecules are arranged in layers. In each layer, molecule are oriented along a privileged direction. This direction varies from one layer to another, so that a helicoidal structure is formed.

Smectic \index{smectic} phase is more ordered : molecules are arranged in layers (see figure figsmectiqueA). The fluid character comes from the ability of layers to slide over their neighbours.

figsmectiqueA

File:SmectiqueA
Smectic A

To describe deformations of nematics, a field of vector is used. To describe deformations of smectics, each state is characterised by set of functions ${\displaystyle u_{i}(x,y)}$ that describes the surface of the i${\displaystyle ^{th}}$ layer. Energical properties of nematic materials are presented at section secenernema.

## Colloids

Colloids \index{colloid} are materials finely divided and dispersed. Examples are emulsions \index{emulsion} and aerosols. Consider a molecule that has two parts: a polar head which is soluble in water and an hydrophobe tail. Such molecules are called amphiphile\index{amphiphile molecule}. Once in water, they gather into small structures called micelles\index{micelle} (see figure fighuileeau). Molecules turn their head to water and their tail to the inside of the structure.

Remark: Cells of live world are very close to micelles, and it may be that life\index{life (origin of)}\index{origin of life} appeared by the mean of micelles.

fighuileeau

File:Huileeau
Amphiphile molecules (as oil molecules) that contains a hydrophile head and an hydrophobe tail organize themselves into small structures called micelles.

Micelles can be encountered in mayonnaise ([#References|references]). Tensioactive substances allow to disolve micelles (application to soaps). Foams are similar arrangements ([#References|references]). For a mathematical introduction to soap films and minimal surfaces, check ([#References|references]).

secglassyspin

## Glass

Glass state ([#References|references]) is characterized by a random distribution of molecules (see figure figglass). Glass\index{glass} is solid, that implies that movements of different constituents are small.

figglass

File:Glass
Glass is characterized by a great rigidity as crystals. However position of atoms are random.}

Phase transition between liquid state and glassy state is done progressively. A material close to glass can be made by compressing small balls together. Under pressure forces, those small balls are deformed and stick to each other. Following question arises: are remaining interstices sufficiently numerous to allow a liquid to pour trough interstices ? This pouring phenomena is a particular case of percolation phenomenon\index{percolation} ([#References|references]). Figure ---figpercol--- illustrates this phenomenom in an experiment presented in ([#References|references]).

figpercol

File:Percol
Example of percolation : black balls are conducting and white balls are insulating. Current running trough the circuit is measured as a function of proportion ${\displaystyle p}$ of white balls. There exists a critical proportion ${\displaystyle p_{c}}$ for which no more current can pass. Study of this critical point allows to exhibit some universal properties encountered in similar systems.

Another example is given by the vandalized grid ([#References|references]) where connections of a conducting wire is destroyed with probability ${\displaystyle p}$.

Spin glasses are disordered magnetic materials. A good example of spin glass is given by the alloy coper manganese, noted CuMn where manganese atoms carrying magnetic moments are dispersed at random in a coper matrix. two spins tend to orient them selves in same or in opposite direction, depending on distance between them (see figure ---figspinglass---).

figspinglass

File:Spinglass
In a spin glass, interactions between neighbour spins are at random ferromagnetic type or anti ferromagnetic type, due to the random distance between spin sites.

The resulting system is called "frustrated": there does not exist a configuration for which all interaction energies are minimal. The simplest example of frustrated system is given by a system constituted by three spins labelled 1,2 and 3 where interactions obey the following rule: energy decreases if 1 and 2 are pointing in the same direction, 2 and 3 are pointing in the same direction and 1 and 3 are pointing in opposite direction. Some properties of spin glasses are presented at section secverredespi.

## Sand piles, orange piles

Physics of granular systems is of high interest and is now the subject of many researches. Those systems, as a sand pile, exhibit properties of both liquids and solids. The sand in a hour-glass doesn't pour like liquids: the speed of pouring doesn't depend on the high of sand above the hole. The formation of the sand pile down of the hole is done by internal convection and avalanche \index{avalanche} at the surface of the pile (see for instance ([#References|references])).