text:
# Publishing Dropwizard Metrics to Kafka

This post is about combining Dropwizard metrics with Kafka to create self instrumenting applications producing durable streams of application metrics, which can be processed (and re-processed) in many ways. The solution is appealing because Kafka is increasingly popular, and therefore likely to be available infrastructure, and Dropwizard metrics likewise, being leveraged by many open source frameworks with many plugins for common measurements such as JVM and web application metrics.

#### DropWizard

Dropwizard metrics allows you to create application metrics as an aspect of your application quickly. An application instrumented  with Dropwizard consists of a MetricRegistry – basically an in memory key-value store of the state of named metrics – and one or more Reporters. There are several built in reporters including ConsoleReporter, CsvReporterGangliaReporter and GraphiteReporter (the Ganglia and Graphite reporters require that you are actually running these services). An unofficial reporter designed for Ambari Metrics is hosted here.  Nobody really wants to work with JMX anymore, but, just in case you’re working with prehistoric code, there is also a JMXReporter available out of the box. Reporters are very loosely coupled with instrumentation cut points throughout your code, so it’s very easy to change a reporting strategy. Instrumenting an application manually is extremely simple – you just can’t go wrong following the getting started page – and there are several annotation processing mechanisms for instrumenting methods; for instance there are numerous integrations to be found on Github for frameworks like Spring. Indeed, I wrote my own annotation binding using Guice type listeners on a recent project, which was certainly easy enough (using techniques in this post on type listeners).

#### Kafka

The only work that needs to be done is to extend the Reporter mechanism to use Kafka as a destination. Despite being fast, the real beauty of writing metrics to Kafka is that you can do what you want with them afterwards. If you want to replicate them real time onto ZeroMQ topics, you can do that just as easily as you can run Spark Streaming or a scheduled Spark Batch job over your application metrics. If you’re building your own monitoring dashboard, you could imagine having a real time latest value, along with hourly or daily aggregations. In fact you can process the metrics at whatever frequency you wish within Kafka’s retention period. I truly believe your application metrics belong in Kafka, at least in the short term.

#### Extending ScheduledReporter

The basic idea is to extend ScheduledReporter composing a KafkaProducer. ScheduledReporter is unsurprisingly invoked repeatedly at a specified rate. On invocation, the idea is to loop through all gauges, meters, timers, and so on, serialise them (there may be a performance boost available from CBOR), and send them to Kafka via the KafkaProducer on a configurable topic. Then wherever in your application you would have created, say, an Slf4jReporter, just create a KafkaReporter instead.

#### Code

<dependency>
<groupId>io.dropwizard.metrics</groupId>
<artifactId>metrics-core</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.0</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.6</version>
</dependency>

Whether you like them or not, all metrics reporters come with builders, so to be consistent you need to implement one. The builder needs to collect some details about Kafka so it knows where to send the metrics. The reporter is going to be responsible for creating a format in this example, but that can be factored out, in which case it would need to be exposed on the builder. In common with all reporters, there are configuration parameters relating to default units etc. which must be exposed for the sake of consistency.

public static class KafkaReporterBuilder {

private final MetricRegistry registry;
private final KafkaProducer<String, byte[]> producer;
private final String topic;
private String name = "KafkaReporter";
private TimeUnit timeUnit = TimeUnit.MILLISECONDS;
private TimeUnit rateUnit = TimeUnit.SECONDS;
private ObjectMapper mapper;

public KafkaReporterBuilder(MetricRegistry registry,
KafkaProducer<String, byte[]> producer,
String topic) {
this.registry = registry;
this.producer = producer;
this.topic = topic;
}

public KafkaReporterBuilder withName(String name) {
this.name = name;
return this;
}

public KafkaReporterBuilder withTimeUnit(TimeUnit timeUnit) {
this.timeUnit = timeUnit;
return this;
}

public KafkaReporterBuilder withRateUnit(TimeUnit rateUnit) {
this.rateUnit = rateUnit;
return this;
}

public KafkaReporterBuilder withMapper(ObjectMapper mapper) {
this.mapper = mapper;
return this;
}

public KafkaReporter build() {
return new KafkaReporter(registry,
name,
MetricFilter.ALL,
rateUnit,
timeUnit,
mapper == null ? new ObjectMapper() : mapper,
topic,
producer);
}
}

Here we will use the metric name as the key of the message, this is because we need all messages of the same metric to go to the same partition to guarantee chronological order. Here we take a KafkaProducer with String keys and byte[] values – the name will be the key, the serialised metric will be the value. It’s better for testability to defer the construction of the KafkaProducer to the caller, so the producer can be mocked, but KafkaProducers are really easy to construct from properties files, for instance see the Javadoc.

The next step is to implement the reporter.

public class KafkaReporter extends ScheduledReporter {

private final KafkaProducer<String, byte[]> producer;
private final ObjectMapper mapper;
private final String topic;

protected KafkaReporter(MetricRegistry registry,
String name,
MetricFilter filter,
TimeUnit rateUnit,
TimeUnit durationUnit,
ObjectMapper mapper,
String topic,
KafkaProducer<String, byte[]> producer) {
super(registry, name, filter, rateUnit, durationUnit);
this.producer = producer;
this.mapper = mapper;
this.topic = topic;
}

@Override
public void report(SortedMap<String, Gauge> gauges,
SortedMap<String, Counter> counters,
SortedMap<String, Histogram> histograms,
SortedMap<String, Meter> meters,
SortedMap<String, Timer> timers) {
report(gauges);
report(counters);
report(histograms);
report(meters);
report(timers);
}

private void report(SortedMap<String, ?> metrics) {
metrics.entrySet()
.stream()
.map(kv -> toRecord(kv.getKey(), kv.getValue(), this::serialise))
.forEach(producer::send);
}

private <T> ProducerRecord<String, byte[]> toRecord(String metricName, T metric, Function<T, byte[]> serialiser) {
return new ProducerRecord<>(topic, metricName, serialiser.apply(metric));
}

private byte[] serialise(Object value) {
try {
return mapper.writeValueAsBytes(value);
} catch(JsonProcessingException e) {
throw new RuntimeException("Value not serialisable: " + value, e);
}
}
}

To use it to publish all application metrics to Kafka in CBOR format, once every five seconds:

MetricRegistry registry = ...
Properties kafkaProperties = ...
KafkaProducer<String, byte[]> producer = new KafkaProducer<>(properties);
KafkaReporter reporter = new KafkaReporter.KafkaReporterBuilder(registry, producer, "topic")
.withMapper(new ObjectMapper(new CBORFactory()))
.build();
reporter.start(5, TimeUnit.SECONDS);
...
reporter.stop();

# A Quick Look at RoaringBitmap

This article is an introduction to the data structures found in the RoaringBitmap library, which I have been making extensive use of recently. I wrote some time ago about the basic idea of bitmap indices, which are used in various databases and search engines, with the caveat that no traditional implementation is optimal across all data scenarios (in terms of size of the data set, sparsity, cardinalities of attributes and global sort orders of data sets with respect to specific attributes). RoaringBitmap is a dynamic data structure which aims to be that one-size-fits-all solution across all scenarios.

#### Containers

A RoaringBitmap should be thought of as a set of unsigned integers, consisting of containers which cover disjoint subsets. Each subset can contain values from a range of size $2^{16}-1$, and the subset is indexed by a 16 bit key. This means that in the worst case it only takes 16 bits to represent a single 32 bit value, so unsigned 32 bit integers can be stored as Java shorts. The choice of container size also means that in the worst case, the container will still fit in L1 cache on a modern processor.

The implementation of the container covering a disjoint subset is free to vary between RunContainer, BitmapContainer and ArrayContainer, depending entirely on properties of the subset. When inserting data into a RoaringBitmap, it is decided whether to create a new container, or to mutate an existing container, depending on whether the values fit in the range covered by the container’s key. When performing a set operation, for instance by intersecting two bitmaps or computing their symmetric difference, a new RoaringBitmap is created by performing operations container by container, and it is decided dynamically which container implementation is best suited for the result. For cases where it is too difficult to determine the best implementation automatically, the method runOptimize is available to the programmer to make sure.

When querying a RoaringBitmap, the query can be executed container by container (which incidentally makes the query naturally parallelisable, but it hasn’t been done yet), and each pair from the cartesian product of combinations of container implementations must be implemented separately. This is manageable because there are only three implementations, and there won’t be any more. There is less work to do for symmetric operations, such as union and intersection, than with asymmetric operations such as contains.

#### RunContainer

When there are lots of clean words in a section of a bitmap, the best choice of container is run length encoding. The implementation of RunContainer is simple and very compact. It consists of an array of shorts (not ints, the most significant 16 bits are in the key) where the values at the even indices are the starts of runs, and the values at the odd indices are the lengths of the respective runs. Membership queries can be implemented simply using a binary search, and quantile queries can be implemented in constant time. Computing container cardinality requires a pass over the entire run array.

#### ArrayContainer

When data is sparse within a section of the bitmap, the best implementation is an array (short[]).  For very sparse data, this isn’t theoretically optimal, but for most cases it is very good and the array for the container will fit in L1 cache for mechanical sympathy. Cardinality is very fast because it is precomputed, and operations would be fast in spite of their precise implementation by virtue of the small size of the set (that being said, the actual implementations are fast). Often when creating a new container, it is necessary to convert to a bitmap for better compression as the container fills up.

#### BitmapContainer

BitmapContainer is the classic implementation of a bitset. There is a fixed length long[] which should be interpreted bitwise, and a precomputed cardinality. Operations on BitmapContainers tend to be very fast, despite typically touching each element in the array, because they fit in L1 cache and make extensive use of Java intrinsics. If you find a method name in here and run your JVM on a reasonably modern processor, your code will quickly get optimised by the JVM, sometimes even to a single instruction. A much hackneyed example, explained better by Michael Barker quite some time ago, would be Long.bitCount, which translates to the single instruction popcnt and has various uses when operating on BitmapContainers. When intersecting with another container, the cardinality can only decrease or remain the same, so there is a chance an ArrayContainer will be produced.

#### Examples

There is a really nice Scala project on github which functions as a DSL for creating RoaringBitmaps – it allows you to create an equality encoded (see my previous bitmap index post) RoaringBitmap in a very fluid way. The project is here.

I have implemented bit slice indices, both equality and range encoded, in a data quality tool I am building. That project is hosted here. Below is an implementation of a range encoded bit slice index as an example of how to work with RoaringBitmaps.

public class RangeEncodedOptBitSliceIndex implements RoaringIndex {

private final int[] basis;
private final int[] cumulativeBasis;
private final RoaringBitmap[][] bitslice;

public RangeEncodedOptBitSliceIndex(ProjectionIndex projectionIndex, int[] basis) {
this.basis = basis;
this.cumulativeBasis = accumulateBasis(basis);
this.bitslice = BitSlices.createRangeEncodedBitSlice(projectionIndex, basis);
}

@Override
public RoaringBitmap whereEqual(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
int[] expansion = expand(code, cumulativeBasis);
for(int i = 0; i < cumulativeBasis.length; ++i) {
int component = expansion[i];
if(component == 0) {
result.and(bitslice[i][0]);
}
else if(component == basis[i] - 1) {
result.andNot(bitslice[i][basis[i] - 2]);
}
else {
result.and(FastAggregation.xor(bitslice[i][component], bitslice[i][component - 1]));
}
}
return result;
}

@Override
public RoaringBitmap whereNotEqual(int code, RoaringBitmap existence) {
RoaringBitmap inequality = existence.clone();
inequality.andNot(whereEqual(code, existence));
return inequality;
}

@Override
public RoaringBitmap whereLessThan(int code, RoaringBitmap existence) {
return whereLessThanOrEqual(code - 1, existence);
}

@Override
public RoaringBitmap whereLessThanOrEqual(int code, RoaringBitmap existence) {
final int[] expansion = expand(code, cumulativeBasis);
final int firstIndex = cumulativeBasis.length - 1;
int component = expansion[firstIndex];
int threshold = basis[firstIndex] - 1;
RoaringBitmap result = component < threshold ? bitslice[firstIndex][component].clone() : existence.clone();     for(int i = firstIndex - 1; i >= 0; --i) {
component = expansion[i];
threshold = basis[i] - 1;
if(component != threshold) {
result.and(bitslice[i][component]);
}
if(component != 0) {
result.or(bitslice[i][component - 1]);
}
}
return result;
}

@Override
public RoaringBitmap whereGreaterThan(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
result.andNot(whereLessThanOrEqual(code, existence));
return result;
}

@Override
public RoaringBitmap whereGreaterThanOrEqual(int code, RoaringBitmap existence) {
RoaringBitmap result = existence.clone();
result.andNot(whereLessThan(code, existence));
return result;
}
}


text:
# qSR: a quantitative super-resolution analysis tool reveals the cell-cycle dependent organization of RNA Polymerase I in live human cells

## Abstract

We present qSR, an analytical tool for the quantitative analysis of single molecule based super-resolution data. The software is created as an open-source platform integrating multiple algorithms for rigorous spatial and temporal characterizations of protein clusters in super-resolution data of living cells. First, we illustrate qSR using a sample live cell data of RNA Polymerase II (Pol II) as an example of highly dynamic sub-diffractive clusters. Then we utilize qSR to investigate the organization and dynamics of endogenous RNA Polymerase I (Pol I) in live human cells, throughout the cell cycle. Our analysis reveals a previously uncharacterized transient clustering of Pol I. Both stable and transient populations of Pol I clusters co-exist in individual living cells, and their relative fraction vary during cell cycle, in a manner correlating with global gene expression. Thus, qSR serves to facilitate the study of protein organization and dynamics with very high spatial and temporal resolutions directly in live cell.

## Introduction

### qSR: quantitative Super Resolution analysis software

We have developed qSR, a software package for quantitative super-resolution data analysis. qSR integrates complementary algorithms that together form a unique tool for the quantitative analysis of single molecule based super-resolution—PALM1,2 and STORM3—data from living cells. The input for qSR is a single-molecule localization dataset, and the prior image processing can be performed with popular open-source software like ImageJ4,5,6. qSR readily accepts as inputs the files generated by super-resolution localization plug-ins in ImageJ, including QuickPALM7, or ThunderSTORM8 which are freely available as add-ons to ImageJ.

Recent open software packages integrate tools for visualization, molecular counting and density based clustering9,10,11,12. However, these tools do not readily utilize temporal dynamics of protein clustering in living cells13,14. Thus a major feature in qSR, which to our knowledge has not been present in any previous analytical package9,10,11,12, is the integrated toolset to analyze the temporal dynamics underlying live cell super-resolution data.

In qSR, we have added some established complementary algorithms for pair-correlation analysis and spatial clustering15,16,17,18 which we found most useful while performing temporal dynamic analyses. One example includes a new application of FastJet19,20,21, a cluster analysis package developed by the particle physics community.

We first test qSR on live cell localization data of endogenously labeled RNA Polymerase II (Pol II) in mouse embryonic fibroblasts, which is known to form transient clusters22 [Fig. 1(a)]. We labeled Pol II by fusing Dendra223, a green-to-red photo-convertible fluorescent protein, to the N terminus of RPB1, the largest subunit of Pol II. The pointillist data obtained from single-molecule based super-resolution microscopy techniques—such as photoactivated localization microscopy (PALM)1,2, stochastic optical reconstruction microscopy (STORM)3 and direct STORM24—can be imported into qSR for visualization and analysis [Fig. 1(b)]. Super-resolution images can be reconstructed, and represented in a red-hot color-coded image, by convolving the point pattern of detections with a Gaussian intensity kernel corresponding to the localization uncertainty [Fig. 1(c)].

In addition, qSR enables the quantitative analysis of the spatial distribution of localizations. The qSR analysis tools provide the user with both a summary of detected clusters, including their areas and number of detections, and a global metric of the distribution of sizes via the pair correlation function. For identifying spatial clusters, we have implemented both centroid-linkage hierarchical clustering using FastJet19,20,21 illustrated in Fig. 1(f), and density-based spatial clustering of applications with noise (DBSCAN)25 as illustrated in Fig. 1(e).

qSR adopts time-correlated super-resolution analyses—for example tcPALM13,14,26,27—to measure the dynamics of sub-diffractive protein clustering in living cells. In live cell super-resolution data, when clusters assemble and disassemble dynamically, the plots of the temporal history of localizations in a cluster show temporal bursts of localizations [Fig. 1(g–i)]. The apparent cluster lifetime and burst size can then be measured, and other clustering parameters, including clustering frequency, can be calculated13,14.

For a sample data set, and step by step instruction on how to perform tcPALM please see the user’s guide in the Supplementary Information, section B.1. It is important to ensure that apparent bursts of detections are not due to long-lived single molecules. Therefore, at minimum, control experiments with fixed cells expressing the fluorophore alone (i.e. unfused to any other protein) should be performed to characterize the temporal profile of individual molecules (Supplementary Information section B.3). We note that there are other super-resolution approaches, such as STED28,29, and structured illumination microscopy30,31 that are not fundamentally based on single-molecule localizations, and the current implementation of qSR is not adapted for those datasets.

### Organization and dynamics of RNA Polymerase I in live human cells

To test the facility of qSR to enable novel quantitative studies with high resolution in living cells we sought to investigate the spatiotemporal organization of RNA Polymerase I (Pol I). Pol I is the molecular complex responsible for the production of ribosomal RNA.

The organization and dynamics of molecular processes inside the nucleolus, the sub-nuclear domain dedicated to ribosome biosynthesis, are emerging as a key intersection point for the regulation of many cellular functions. To balance the need for cellular growth with the energetic cost of ribosome biosynthesis, cells have evolved complex regulatory mechanisms to modulate ribosomal biogenesis in response to the availability of nutrients and growth factors32,33. Similarly, stresses that affect cellular homeostasis generally result in signaling cascades that regulate ribosome biogenesis by controlling transcription by Pol I34,35,36. Pol I is thus thought to be a promising target for the treatment of tumors37,38 given its central position at the convergence of many different pathways controlling cell proliferation and stress. Yet, despite the central position of Pol I regulation in coordination of cellular processes, it remains unclear how Pol I organization and dynamics may depend on, or reflect cellular demand for ribosomal RNA transcription.

Previous studies in living cells39 using fluorescence recovery after photobleaching (FRAP) and kinetic modeling on FRAP data suggested the existence of multiple subpopulations of Pol I interactions with the ribosomal genes. In fact, further studies suggested that ribosomal transcriptional regulation occurs via cell-cycle dependent modulation of these kinetics by altering the availability of a transcription factor (SL1) thereby changing the assembly efficiency of the ribosomal transcription complex40. Ensemble-averaging techniques like FRAP, however, are not readily amenable to direct visualization of co-existing dynamic subpopulations and characterization of how each subpopulation putatively changes to help balance cellular progression through different cycles.

Here we use single-molecule based super-resolution to directly measure dynamics of endogenous Pol I in living cells. We fused the catalytic subunit of Pol I, RPA194, to a photoconvertible fluorescent protein, Dendra223, using the CRISPR/Cas-9 genome editing system22,41,42,43,44 in a human osteosarcoma (U2OS) cell line to create a heterozygous, endogenously labeled cell line. Upon illumination with 405 nm light, the emission maximum of Dendra2 converts from green to red23 allowing for imaging of single RPA194 molecules in live cells. We employed quantitative super-resolution imaging on this endogenously labeled Pol I cell line to investigate Pol I organization and dynamics in live cells13,14.

With this endogenously labeled Pol I cell line, we used time-correlated photoactivated localization microscopy (tcPALM) to investigate the real time, super-resolution dynamics of Pol I13,14. In single-molecule localization based super-resolution techniques such as PALM or STORM a final image is reconstructed based on precise localizations of a sparse, stochastically activated set of fluorophores1,2,3. Time-correlated super-resolution analysis extracts protein dynamics from a super-resolution reconstruction. Since the number of localization events in a temporal window can serve as a measure of local concentration (provided a sufficiently large number of molecules activates within the window), super-resolution acquisition in live cells can provide a measure of the relative spatial and temporal fluctuations of the labeled proteins.

## Results

### Investigation of Pol I cluster dynamics in the interphase nucleus

We began by investigating the real-time dynamics of Pol I during interphase. We imaged a total of 77 individual cells from our Dendra2-RPA194 cell line on seven separate days. Consistent with previous observations39 we find that Pol I is localized in distinct foci within the nucleoli. Many of these foci are directly visible by conventional live cell imaging [Fig. 2(b)].

Examining the loci with live cell super-resolution imaging [Fig. 2(c)] and quantifying the dynamics of individual loci using tcPALM reveals that these foci are stable [Fig. 2(d)]. The presence of a cluster is evident in a tcPALM time trace as a steady stream of localizations in the number of detection per frame, or a slope in the cumulative plot. For a stable cluster, the slope is highest from the beginning acquisition (t = 0), indicating that the cluster existed even before image acquisition started. Then, a gradual plateau in the tcPALM time trace suggests that the cluster is still present, but the pool of photoconvertible fluorescent proteins is being gradually depleted by activation and bleaching during the imaging process. The timescale of photoactivation (minutes) determines how quickly the pool of fluorescent proteins in depleted. Consequently, clusters that disassemble more slowly that the activation rate will appear stable in tcPALM.

The presence of multiple, stable sub-nucleolar Pol I clusters during interphase is consistent with a picture of the interphase nucleolus sub-organized into distinct regions including punctate dense transcriptional centers (with high Pol I concentration) separated by a surrounding granular component45,46,47,48.

### Dynamics of Pol I clustering upon transcription inhibition with CX-5461

Next, we sought to investigate Pol I dynamics at low transcription levels using drug inhibition. We inhibited Pol I initiation with a small molecule inhibitor CX-5461. CX-5461 is a synthetic compound that selectively inhibits Pol I without affecting other transcription (for example mRNA transcription by RNA Polymerase II)49,50. In the canonical Pol I recruitment pathway, the factor UBF binds to the rDNA promoter and recruits SL132. SL1 then recruits the downstream transcription factors and associated machinery needed to bind Pol 1 to the promoter. CX-5461 is thought to act selectively on Pol 1 by inhibiting SL1, thereby preventing Pol I initiation. The drug has additionally been observed to induce autophagy and prevent cell growth and division.

We incubated the cells in 2 μM CX-5461 for 48 hours before performing live cell super-resolution experiments. We imaged a total of 29 cells treated with CX-5461 on three separate days. The initiation-inhibited cells also show clusters in the super-resolution reconstruction (Fig. 3A).

The time traces of individual clusters in drug-treated cells, however, are highly dynamic. In contrast to the stable clusters observed in untreated cells (Fig. 2), many of the initiation-inhibited clusters are transient [Fig. 3(b)] while others are stable (Supplementary Fig. S2). In each cell, we saw a mix of both transient and stable clusters. We can identify a region of interest where a cluster existed at some point during acquisition by examining the final PALM image. In tcPALM localization time traces of these regions, transient clusters are evident by time traces characterized with virtually no localizations at the beginning, suggesting that the cluster had not yet formed; then localization frequency suddenly increases, indicating a sudden increase in local concentration of photoconvertible proteins, and suggesting the cluster assembled in real time (for more information on the transient/stable distinction, see Supplementary Fig. S3). The high frequency of localization then ceases abruptly suggesting that the cluster has rapidly disassembled [Fig. 3(b)]. The number of detections in this transient jump cannot be accounted for by single molecule photophysics, so multiple polymerases multiple proteins were likely recruited to the locus13,51,52. Together, the features in the example time trace in Fig. 3(b) suggest that Pol I clusters become transient, assembling and disassembling dynamically, upon transcription inhibition. The transient Pol I clusters we observe have a mean lifetime of 8.7 ± 0.7 s [Fig. 3(d)].

To ensure that the transient clusters we observed were in fact distinct from the stable Pol I foci, and not simply small, but stable, Pol I clusters, we performed simulations of Dendra2 photo-physics. We compared the progression of polymerases in a stable cluster to a cluster of polymerases that only associated for a brief time. The details of our simulation scheme can be found in Materials and Methods. A comparison of the results of our simulation for both the stable clusters we observed in interphase and the transient clusters we observe when we inhibit initiation with CX-5461 can be found in Supplementary Fig. S2. Our simulations suggest that transient features observed in the tcPALM traces are not compatible with stable foci given the photophysics of Dendra2.

We quantified the fraction of transient clusters in CX-5461 treated and control cells [Fig. 3(c)]. We observe that the drug treated cells contain substantially more transient clusters than the control cells. Remaining stable clusters may result from the drug failing to turning off all ribosomal transcription; consistent with this interpretation we find that the remaining stable clusters are similar in size to those found in the control cells [Fig. 3(e)] with no significant difference in the distributions.

### Cell-cycle dependence of Pol I clustering

We investigated the dynamics of Pol I clusters in various stages of the cell cycle. Since rDNA transcription ceases when chromosomes are condensed, we were curious if we would observe a similar increase in transient clustering during mitosis. We hoped to then observe how clustering dynamics changes as cells grew and replicated their DNA in preparation for division. We blocked cells in S phase by halting the cell cycle using a double thymidine block53. Releasing these cells from thymidine produces synchronized cells in M and G1 phases. We imaged 29 M phase cells, 27 G1 phase cells and 30 S phase cells on nine separate days.

The Pol I spatial organization in M phase differs substantially from interphase Pol I organization. Unlike in interphase where we observed foci of multiple, adjacent Pol I clusters in nucleoli, we see no similar groupings of Pol I clusters in M phase. In fact, we observe fewer and spatially distinct Pol I clusters in M phase [Fig. 4(a)]. tcPALM analysis of cells imaged in M Phase shows a mixture of both stable and transient Pol I clusters [Fig. 4(b)].

The observation of a mixture of stable and transient Pol I clusters agrees well with previous descriptions of mitotic rDNA organization32,45,54. During mitosis, the chromosomes condense and transcription of rDNA ceases55. Previously bound Pol I molecules are thought to remain bound to the rDNA genes during this transcriptionally silent period45. Novel Pol I binding events, however, are believed to be inhibited, putatively through Runx2/SL1 dependent transcription inhibition54. The stable clusters in M phase are likely attributable to previously initiated and elongating polymerases that remain bound to the rDNA genes during mitosis. The transient clusters, however, may be due to the absence of the SL1 as we observed in our drug inhibition experiments in Fig. 3.

To quantify the differences in Pol I cluster dynamics between different phases of the cell cycle, we evaluated both the portion of transient and of stable clusters and the size distribution of the stable clusters [Fig. 4(b)]. We found that M phase displays the highest fraction of transient clusters, and the fraction of transient clusters decreases as cells progress into G1 and later S phase. Similarly, we quantified the relative intensity of stable clusters.

We observe more detections per stable clusters as cells progress out of M phase into G1 and S (Fig. 4(c) and Supplementary Fig. S4). Translating number of detections into number of molecules is an intricate task in super-resolution data due to non-trivial single-molecule photophysics. Nonetheless, more detections likely indicate that more polymerase molecules are present in the clusters. The cluster size distribution as measured by the number of localization from the stable clusters between M, G1 and S phase was noticeably different [compare different color graphs in Fig. 4(b)]. Thus, Pol I clusters tend to become more intense (i.e., larger by the number of molecules per foci) and more stable as the cells grow and synthesize new DNA in preparation for division.

We note that the observation of larger, more stable clusters corroborates previous interpretations from FRAP data showing longer retention times for individual polymerase subunits and the SL1 transcription factor in S phase than G140. The increase in Pol I cluster stability and size also agrees with previous observations that Pol I activity increases as cells progress through G1 and peaks in S phase35.

The simultaneous presence of a distinct transient population of RNA Polymerase I, however, could not have been directly inferred from previous measurements. Moreover, our observation of the onset of transient clustering from both drug-induced and cell-cycle driven inhibition of SL1 suggests that the fraction of stable versus transient clusters may be interpretable as the relative fraction of actively transcribed rDNA genes versus inactive, silent genes. This measure thus likely reflects the overall activity of the rDNA genes.

## Conclusion

Taken together, our live cell super-resolution data paint a dynamic picture of cell cycle dependent Pol I clustering: Pol I forms stable foci in the nucleolus where rDNA genes are clustered and when many polymerases are actively transcribing the rDNA genes. When initiation is inhibited, Pol I may still cluster, albeit very transiently. Transcriptionally active nucleolar centers thus appear as stable clusters of polymerase while inactive centers appear as transient clusters.

The type of quantitative live cell super-resolution analysis can be done in principle for any protein that can be fused to a GFP-like fluorescent tag. The real time quantitative characterizations of Pol I, and the direct observation of different subpopulations (transient versus stable) of Pol I clusters in living cells, serves to illustrate the types of novel insights that can be drawn from their own studies using the qSR analytical software.

qSR is provided as an open source software package with a graphical user interface that a researcher can download and use directly. For illustration, and to help new users quickly familiarize themselves with qSR, we provide the live cell super-resolution data represented in Fig. 1 as sample data22 packaged along with the software, as well as step by step instructions in Doc. S1. Installation instructions and protocols for performing the analyses are also provided in Doc. S1. qSR is maintained and distributed for free on www.github.com/cisselab/qSR. By providing qSR as an open source platform, we hope to encourage other users in the community to develop and integrate any multitude of algorithms, features and modules for qSR, as the field evolves. For examples approaches for counting molecules in clusters51,56,57, could be complimentary extensions beyond the current scope of qSR. Users are prompted to directly cite the original publications for any third-party algorithm they use within the qSR platform.

## Materials and Methods

### CRISPR-Cas9 mediated insertion of Dendra2 onto the N-Terminus of RPA194

A human osteosarcoma (U2OS) cell line with an endogenous N-terminal Dendra2 insertion in the RPA194 gene was generated via the CRISPR/Cas9 system41,42,43,44.

The cells were transfected with both pSpCas9(BB)-2A-Puro (px459) V2.0, containing an sgRNA targeted to start of the RPA194 gene (5′-TTCAGCCGAATACATCCCCGAAGG-3′), and a homology directed repair template. sgRNA sequences were generated via the online CRISPR toolbox (crispr.mit.edu) and then cloned into the px459 vector with one step digestion ligation (Supplementary Table S1)54,58. All experiments and validation were performed on the cell line transfected with sgRNA1. Of the remaining guides, only sgRNA5 showed successful insertion. pSpCas9(BB)-2A-Puro (px459) V2.0 was a gift from Feng Zhang (Addgene plasmid # 62988).

The homologous donor sequence was synthesized by Life Technologies to contain a 500 bp left homologous arm, the 690 bp Dendra2 insertion and a 500 bp right homologous arm. Silent mutations were introduced into the protospacer adjacent motif (PAM) sites of 6 potential sgRNAs in the right homologous arm to ensure the repair template was not degraded by the Cas9 system. The repair template was PCR amplified to prepare a linear fragment for transfection (Supplementary Table S2).

The linear homologous repair template and the px459 plasmid containing the sgRNA insert were transfected into the U2OS cell line using the x-tremeGENE 9 transfection reagent from Sigma-Aldrich. Cells were left to incubate at 37 °C for 24 hours. The cells were then allowed to recover, and sorted via FACS at the Koch Institute Sorting facility to isolate cells expressing the Dendra2 insertion for imaging.

### Cell Culture Protocols

The Dendra2-Pol I cells and Dendra2-Pol II cells22 were cultured in Dulbecco’s Modified Eagle Medium with Glutamax (DMEM with Glutamax) from Thermo Fisher (10567). The media contained 10% fetal bovine serum from Gibco (26140-079, US Origin, Qualified) and an antibiotic mixture at a final working concentration of 10 U/mL penicillin and 10 μg/ml streptomycin from Gibco (15140). Cells were incubated at 37 °C with 5% supplemental CO2.

### Initiation Inhibition with CX-5461

Cells were incubated with 2 μM CX-5461 (Selleckchem 1138549-36-6). CX-5461 is a potent, selective inhibitor of SL1, a transcription factor associated with Pol I binding to the rDNA gene49,50. CX-5461 was stored at 10 mM in a stock solution of 50 mM NaH2PO4 (pH 4.5) and added directly to imaging dishes for treatment.

### Cell Cycle Synchronization via a Double Thymidine Block

Cells were synchronized using a double thymidine block approach previously reported53. Cells were treated with 2 mM of thymidine from Sigma Aldrich (T1895-1G) dissolved in the 10% FBS DMEM media previously described and incubated at 37 °C for 15 hours to arrest cells in S phase. Following this initial S phase arrest, cells were released via removal of the thymidine medium and allowed to progress for 9 hours to ensure all cells had passed out of S phase. Cells were then re-incubated with 2 mM thymidine to synchronize cells at the G1/S junction. To produce cells in S phase, cells were imaged ~one hour after release from thymidine control. For M phase, cells were imaged between 9 and 12 hours after release from thymidine and subject to visual confirmation of ongoing mitosis (i.e., rounded cell shape, finger-like projections, cell division, etc.). For G1 cells, cells were imaged 15 hours after thymidine release.

### Super-Resolution Imaging

Cells were imaged on a homebuilt super-resolution setup, comprised of a Nikon Eclipse TI microscope equipped with a 100x oil immersion objective (NA 1.40) (Nikon, Tokyo, Japan) and lasers and filter sets for 405 nm, 488 nm, and 561 nm illumination. During imaging, cells were kept at 37 °C in an incubator set atop the objective (InVivo Scientific, St. Louis, MO). Images were captured on an Andor iXon Ultra 897 EMCCD camera at a rate of 60 ms/frame at an EM-gain of 900. Camera image acquisition and control was performed using Micro Manager 1.459. Cells were held steady in the z-direction using the Perfect Focus System of the Nikon Microscope during image acquisition. Conventional fluorescence imaging was achieved via 50 ms of exposure with a 488 nm laser.

For PALM imaging, excitation (561 nm) and activation (405 nm) lasers were combined, expanded then focused on the sample. These beams were expanded using an achromatic beam expander (AC254-040-A and AC508-300-A, from THORLABS, Newton, NJ) and refocusing was performed with an achromatic converging lens (#45–354, from Edmund Optics, Barrington, NJ). Power levels were controlled both by directly varying the initial laser intensity and through an AOTF, and measured directly at the top of the objective lens closest to the sample slide.

### Image Analysis and tcPALM

We analyzed images following the general scheme previously described in depth in Cho et al. and Cissé et al.13,14. Individual Pol I molecules were localized using a modified MTT localization algorithm60. Super-resolution images were generated from a Gaussian spreading of the localizations determined by the MTT program. We then analyzed these localizations using qSR. Step-by-step instructions of how to use qSR can be found in Supplementary Information.

### tcPALM Simulation Using Hidden Markov Models

We simulated tcPALM acquisitions using the hidden Markov model package in MATLAB61. As in previous work, we considered a four-state model (pre-converted, fluorescent, dark and bleached) for Dendra222,51. Initially, all of the Dendra2 fluorophores are in a dark, pre-converted states. As acquisition proceeds, the Dendra2 are stochastically photoactivated by the 405 nm laser at a rate kon. This photoactivation converts Dendra2 into a fluorescent state. Since Dendra2 is a blinking dye, this fluorescent state exists in dynamic equilibrium with a dark state distinct from the bleached state. The fluorescent state can convert to the dark state with a rate kdark and recover back to the fluorescent state with rate krec. In contrast, photobleaching is irreversible. The fluorescent state is then photobleached by the 561 nm laser at a rate kbleach and proceeds, into the bleached state. Of these four states, only the post-conversion fluorescent state is directly observable during tcPALM acquisition.

Previous work in our lab and others13,51,52 has assessed the photophysical parameters of Dendra2 in U2OS. Since we engineered our cells from the same stock and used the same microscope setup with the same 561 nm laser power, we felt confident using the previously determined values for kdark, krec and kbleach (kdark = 9.61/s, krec = 2.33/s and kbleach = 3.0/s). We did, however, use a higher photoactivation power than previous experiments, so we measured kon independently. We analyzed the tcPALM trace of the entire nucleus and fit our trace to an exponential model with a constant, linear noise term: N = N0 (1 − exp(−kon * t)) + B * t. From this analysis, we found that kon = (1/43 s).

We assigned each of these four states to a state of a hidden Markov model. Given the progression scheme described above, we used the transition matrix:

$${\rm{T}}=\begin{array}{cccc}[(1-{{\rm{k}}}_{{\rm{o}}{\rm{n}}}\,\ast \,{\rm{\Delta }}t) & {{\rm{k}}}_{{\rm{o}}{\rm{n}}}\,\ast \,{\rm{\Delta }}t & 0 & 0\\ 0 & (1-{{\rm{k}}}_{{\rm{d}}{\rm{a}}{\rm{r}}{\rm{k}}}\,\ast \,{\rm{\Delta }}t-{{\rm{k}}}_{{\rm{b}}{\rm{l}}{\rm{e}}{\rm{a}}{\rm{c}}{\rm{h}}}\,\ast \,{\rm{\Delta }}t) & {{\rm{k}}}_{{\rm{d}}{\rm{a}}{\rm{r}}{\rm{k}}}\,\ast \,{\rm{\Delta }}t & {{\rm{k}}}_{{\rm{b}}{\rm{l}}{\rm{e}}{\rm{a}}{\rm{c}}{\rm{h}}}\,\ast \,{\rm{\Delta }}t\\ 0 & {{\rm{k}}}_{{\rm{r}}{\rm{e}}{\rm{c}}}\,\ast \,{\rm{\Delta }}t & (1-{{\rm{k}}}_{{\rm{r}}{\rm{e}}{\rm{c}}}\,\ast \,{\rm{\Delta }}t) & 0\\ 0 & 0 & \,0 & 1]\end{array}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,$$

We simulated progression of an adjustable number of Dendra2 molecules through the tcPALM acquisition process individually and aggregated the results to simulate cluster dynamics. We used a timestep ∆t of 6 ms and then binned our results into distinct 60 ms bins to simulate our exposure time. If any of the molecules we had simulated were in the fluorescent state at any point in our 60 ms bin, we assumed that we would observe a localization. Within a cluster, if multiple molecules are visible at the same time they will appear as a single spot. Consequently, they will contribute a single localization count. Therefore, in the simulation this is reflected as a single detection count. For stable clusters, we analyzed our simulation result for the entire acquisition window while for transient clusters we only considered our acquisition results within a limited temporal window and set the localizations equal to zero elsewhere.

## References

1. 1.

Betzig, E. et al. Imaging Intracellular Fluorescent Proteins at Nanometer Resolution. Science. 313, 1642–1645 (2006).

2. 2.

Hess, S. T., Girirajan, T. P. K. & Mason, M. D. Ultra-High Resolution Imaging by Fluorescence Photoactivation Localization Microscopy. Biophys. J. 91, 4258–4272 (2006).

3. 3.

Rust, M. J., Bates, M. & Zhuang, X. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Methods 3, 793–796 (2006).

4. 4.

Abràmoff, M. D., Magalhães, P. J. & Ram, S. J. Image processing with imageJ. Biophotonics International 11, 36–41 (2004).

5. 5.

Collins, T. J. ImageJ for microscopy. BioTechniques 43, 25–30 (2007).

6. 6.

Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).

7. 7.

Henriques, R. et al. QuickPALM: 3D real-time photoactivation nanoscopy image processing in ImageJ. Nat. Methods 7, 339–340 (2010).

8. 8.

Ovesný, M., Křížek, P., Borkovec, J., Švindrych, Z. & Hagen, G. M. ThunderSTORM: A comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging. Bioinformatics 30, 2389–2390 (2014).

9. 9.

Andronov, L., Lutz, Y., Vonesch, J. L. & Klaholz, B. P. SharpViSu: Integrated analysis and segmentation of super-resolution microscopy data. Bioinformatics 32, 2239–2241 (2016).

10. 10.

Levet, F. et al. SR-Tesseler: a method to segment and quantify localization-based super-resolution microscopy data. Nat. Methods 12, 1065–1071 (2015).

11. 11.

Pengo, T., Holden, S. J. & Manley, S. PALMsiever: A tool to turn raw data into results for single-molecule localization microscopy. Bioinformatics 31, 797–798 (2015).

12. 12.

Malkusch, S. & Heilemann, M. Extracting quantitative information from single-molecule super-resolution imaging data with LAMA – LocAlization MicroscopyAnalyzer. Sci. Rep. 6, 34486 (2016).

13. 13.

Cho, W.-K. et al. RNA Polymerase II cluster dynamics predict mRNA output in living cells. Elife 5 (2016).

14. 14.

Cisse, I. I. et al. Real-Time Dynamics of RNA Polymerase II Clustering in Live Human Cells. Science. 341, 664–667 (2013).

15. 15.

Sengupta, P., Jovanovic-Talisman, T. & Lippincott-Schwartz, J. Quantifying spatial organization in point-localization superresolution images using pair correlation analysis. Nat. Protoc. 8, 345–354 (2013).

16. 16.

Sengupta, P. et al. Probing protein heterogeneity in the plasma membrane using PALM and pair correlation analysis. Nat. Methods 8, 969–975 (2011).

17. 17.

Veatch, S. L. et al. Correlation functions quantify super-resolution images and estimate apparent clustering due to over-counting. PLoS One 7 (2012).

18. 18.

Annibale, P., Vanni, S., Scarselli, M., Rothlisberger, U. & Radenovic, A. Identification of clustering artifacts in photoactivated localization microscopy. Nat. Methods 8, 527–528 (2011).

19. 19.

Cacciari, M., Salam, G. P. & Soyez, G. FastJet user manual. Eur. Phys. J. C 72, 1896 (2011).

20. 20.

Dokshitzer, Y., Leder, G., Moretti, S. & Webber, B. Better jet clustering algorithms. J. High Energy Phys. 1997, 001–001 (1997).

21. 21.

Wobisch, M. & Wengler, T. Hadronization corrections to jet cross sections in deep-inelastic scattering. arXiv (1999).

22. 22.

Cho, W.-K. et al. Super-resolution imaging of fluorescently labeled, endogenous RNA Polymerase II in living cells with CRISPR/Cas9-mediated gene editing. Sci. Rep. 6, 35949 (2016).

23. 23.

Chudakov, D. M., Lukyanov, S. & Lukyanov, K. A. Tracking intracellular protein movements using photoswitchable fluorescent proteins PS-CFP2 and Dendra2. Nat. Protoc. 2, 2024–2032 (2007).

24. 24.

Heilemann, M. et al. Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes. Angew. Chemie - Int. Ed. 47, (6172–6176 (2008).

25. 25.

Ester, M., Kriegel, H. P., Sander, J. & Xu, X. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. Proc. 2nd Int. Conf. Knowl. Discov. Data Min. 226–231 doi:10.1.1.71.1980 (1996).

26. 26.

Liu, Z. et al. 3D imaging of Sox2 enhancer clusters in embryonic stem cells. Elife 3, e04236 (2014).

27. 27.

Chen, X. et al. Study of RNA Polymerase II Clustering inside Live-Cell Nuclei Using Bayesian Nanoscopy. ACS Nano 10, 2447–2454 (2016).

28. 28.

Hell, S. W. & Wichmann, J. Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Opt. Lett. 19, 780 (1994).

29. 29.

Klar, T. A., Jakobs, S., Dyba, M., Egner, A. & Hell, S. W. Fluorescence microscopy with diffraction resolution barrier broken by stimulated emission. Proc. Natl. Acad. Sci. 97, 8206–8210 (2000).

30. 30.

Gustafsson, M. G. L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J. Microsc. 198, 82–87 (2000).

31. 31.

Gustafsson, M. G. L. Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl. Acad. Sci. 102, 13081–13086 (2005).

32. 32.

Grummt, I. Life on a planet of its own: regulation of RNA polymerase I transcription in the nucleolus. Genes Dev. 17, 1691–702 (2003).

33. 33.

Kusnadi, E. P. et al. Regulation of rDNA transcription in response to growth factors, nutrients and energy. Gene 556, 27–34 (2015).

34. 34.

Boulon, S., Westman, B. J., Hutten, S., Boisvert, F. M. & Lamond, A. I. The Nucleolus under Stress. Molecular Cell 40, 216–227 (2010).

35. 35.

Grummt, I. The nucleolus—guardian of cellular homeostasis and genome integrity. Chromosoma 122, 487–497 (2013).

36. 36.

Zhao, Z., Dammert, M. A., Grummt, I. & Bierhoff, H. lncRNA-Induced Nucleosome Repositioning Reinforces Transcriptional Repression of rRNA Genes upon Hypotonic Stress. Cell Rep. 14, 1876–1882 (2016).

37. 37.

Schneider, D. A. RNA polymerase I activity is regulated at multiple steps in the transcription cycle: recent insights into factors that influence transcription elongation. Gene 493, 176–84 (2012).

38. 38.

Quin, J. E. et al. Targeting the nucleolus for cancer intervention. Biochim. Biophys. Acta - Mol. Basis Dis. 1842, 802–816 (2014).

39. 39.

Dundr, M. et al. A Kinetic Framework for a Mammalian RNA Polymerase in Vivo. Science. 298, 1623–1626 (2002).

40. 40.

Gorski, S. A., Snyder, S. K., John, S., Grummt, I. & Misteli, T. Modulation of RNA Polymerase Assembly Dynamics in Transcriptional Regulation. Mol. Cell 30, 486–497 (2008).

41. 41.

Cong, L. et al. Multiplex Genome Engineering Using CRISPR/Cas Systems. Science. 339, 819–823 (2013).

42. 42.

Jinek, M. et al. RNA-programmed genome editing in human cells. Elife 2, e00471 (2013).

43. 43.

Mali, P. et al. RNA-guided human genome engineering via Cas9. Science 339, 823–6 (2013).

44. 44.

Ran, F. A. et al. Genome engineering using the CRISPR-Cas9 system. Nat. Protoc. 8, 2281–2308 (2013).

45. 45.

Roussel, P., André, C., Comai, L. & Hernandez-Verdun, D. The rDNA transcription machinery is assembled during mitosis in active NORs and absent in inactive NORs. J. Cell Biol. 133, 235–46 (1996).

46. 46.

Tschochner, H. & Hurt, E. Pre-ribosomes on the road from the nucleolus to the cytoplasm. Trends Cell Biol. 13, 255–63 (2003).

47. 47.

Thomson, E., Ferreira-Cerca, S. & Hurt, E. Eukaryotic ribosome biogenesis at a glance. J. Cell Sci. 126, 4815–21 (2013).

48. 48.

Feric, M. et al. Coexisting Liquid Phases Underlie Nucleolar Subcompartments. Cell 165, 1686–1697 (2016).

49. 49.

Drygin, D. et al. Targeting RNA polymerase I with an oral small molecule CX-5461 inhibits ribosomal RNA synthesis and solid tumor growth. Cancer Res. 71, 1418–30 (2011).

50. 50.

Haddach, M. et al. Discovery of CX-5461, the First Direct and Selective Inhibitor of RNA Polymerase I, for Cancer Therapeutics. ACS Med. Chem. Lett. 3, 602–6 (2012).

51. 51.

Lee, S.-H., Shin, J. Y., Lee, A. & Bustamante, C. Counting single photoactivatable fluorescent molecules by photoactivated localization microscopy (PALM). Proc. Natl. Acad. Sci. USA 109, 17436–41 (2012).

52. 52.

Pennacchietti, F., Gould, T. J. & Hess, S. T. The Role of Probe Photophysics in Localization-Based Superresolution Microscopy. Biophys. J. 113, 2037–2054 (2017).

53. 53.

Bostock, C. J., Prescott, D. M. & Kirkpatrick, J. B. An evaluation of the double thymidine block for synchronizing mammalian cells at the G1-S border. Exp. Cell Res. 68, 163–168 (1971).

54. 54.

Young, D. W. et al. Mitotic occupancy and lineage-specific transcriptional control of rRNA genes by Runx2. Nature 445, 442–446 (2007).

55. 55.

Klein, J. & Grummt, I. Cell cycle-dependent regulation of RNA polymerase I transcription: the nucleolar transcription factor UBF is inactive in mitosis and early G1. Proc. Natl. Acad. Sci. USA 96, 6096–101 (1999).

56. 56.

Rollins, G. C., Shin, J. Y., Bustamante, C. & Pressé, S. Stochastic approach to the molecular counting problem in superresolution microscopy. Proc. Natl. Acad. Sci. USA 112, E110–8 (2015).

57. 57.

Fricke, F., Beaudouin, J., Eils, R. & Heilemann, M. One, two or three? Probing the stoichiometry of membrane proteins by single-molecule localization microscopy. Sci. Rep. 5, 14072 (2015).

58. 58.

Hsu, P. D. et al. DNA targeting specificity of RNA-guided Cas9 nucleases. Nat. Biotechnol. 31, 827–832 (2013).

59. 59.

Edelstein, A. D. et al. Advanced methods of microscope control using μManager software. J. Biol. methods 1,

60. 60.

Sergé, A., Bertaux, N., Rigneault, H. & Marguet, D. Dynamic multiple-target tracing to probe spatiotemporal cartography of cell membranes. Nat. Methods 5, 687–694 (2008).

61. 61.

Eddy, S. R. What is a hidden Markov model? Nat. Biotechnol. 22, 1315–1316 (2004).

## Acknowledgements

The work is supported by the National Institutes of Health, the National Cancer Institutes through the NIH Director’s New Innovator Award, DP2-CA195769 to IIC and funds from MIT Department of Physics. Further support for this work was provided by NIH 4D Nucleome program, grant U54-DK107980, the DeFlorez Endowment Fund, and the MIT Undergraduate Research Opportunities Program (UROP). JOA is supported by the National Science Foundation Graduate Research Fellowship. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health nor the National Science Foundation. The software is distributed open-source under the GNU General Public License (GPLv2/LGPLv2) Version 2. qSR is maintained and distributed for free on www.github.com/cisselab/qSR.

## Author information

The software was conceived by J.O.A. and I.I.C. and the Pol I experiments were conceived by W.C. and I.I.C. J.T. provided expertise in spatial clustering and implemented the initial version of FastJet-based spatial clustering. Initial versions of the other algorithms were implemented by J.O.A., A.N., and J.H.S. The rest of the software was developed by J.O.A. T.I. and I.I.C. designed and built the homebuilt super-resolution microscope. W.-K.C., N.J. and S.M. provided assistance in the generation of the endogenous Dendra2-RPA190 cell line. W.-K.C. acquired the Pol II data, and W.C. acquired and analyzed the Pol I data. J.O.A., W.C., W.-K.C., and I.I.C. wrote the manuscript.

Correspondence to I. I. Cissé.

## Ethics declarations

### Competing Interests

The authors declare no competing interests.

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Andrews, J.O., Conway, W., Cho, W.-. et al. qSR: a quantitative super-resolution analysis tool reveals the cell-cycle dependent organization of RNA Polymerase I in live human cells. Sci Rep 8, 7424 (2018) doi:10.1038/s41598-018-25454-0

• ### A first order phase transition mechanism underlies protein aggregation in mammalian cells

• Arjun Narayanan
• , Anatoli Meriin
• , J Owen Andrews
• , Jan-Hendrik Spille
• , Michael Y Sherman
•  & Ibrahim I Cisse

eLife (2019)

• ### Cholesterol modulates acetylcholine receptor diffusion by tuning confinement sojourns and nanocluster stability

• Alejo Mosqueira
• , Pablo A. Camino
•  & Francisco J. Barrantes

Scientific Reports (2018)

• ### Mediator and RNA polymerase II clusters associate in transcription-dependent condensates

• Won-Ki Cho
• , Jan-Hendrik Spille
• , Micca Hecht
• , Choongman Lee
• , Charles Li
• , Valentin Grube
•  & Ibrahim I. Cisse

Science (2018)

format: plain


text:
Home > CCA2 > Chapter 11 > Lesson 11.2.2 > Problem11-54

11-54.

Daylight Saving Time was adopted in the U.S. in $1918$. During the oil crisis of the $1970$s, the Department of Transportation found that daylight saving time decreased national energy usage by about $1$ percent compared with standard time. Since then, energy use in the U.S. has changed and daylight saving time has been extended. In $2007$ a group of researchers found that daylight saving time decreased national energy usage by $0.2\%$ with a margin or error of $1.5\%$. If a $1\%$ decrease in energy use will save the state of Indiana $\9$ million, what conclusion(s) can you draw from the $2007$ study in relation to the state of Indiana?

Energy usage can be expected to change in Indiana by $0.2\%\pm1.5\%$.
What range of percentages is this? Use this range to calculate the cost/savings to the state.


text:
# Chord (aeronautics)

(Redirected from Chord (aircraft))
Chord of an aerofoil section.
Chords on a swept-wing

In aeronautics, a chord is the imaginary straight line joining the leading and trailing edges of an aerofoil. The chord length is the distance between the trailing edge and the point on the leading edge where the chord intersects the leading edge.[1][2]

The point on the leading edge that is used to define the chord can be defined as either the surface point of minimum radius,[2] or the surface point that will yield maximum chord length[citation needed].

The wing, horizontal stabilizer, vertical stabilizer and propeller of an aircraft are all based on aerofoil sections, and the term chord or chord length is also used to describe their width. The chord of a wing, stabilizer and propeller is determined by measuring the distance between leading and trailing edges in the direction of the airflow. (If a wing has a rectangular planform, rather than tapered or swept, then the chord is simply the width of the wing measured in the direction of airflow.) The term chord is also applied to the width of wing flaps, ailerons and rudder on an aircraft.

The term is also applied to aerofoils in gas turbine engines such as turbojet, turboprop, or turbofan engines for aircraft propulsion.

Most wings are not rectangular so they have a different chord at different positions along their span. To give a characteristic figure that can be compared among various wing shapes, the mean aerodynamic chord, or MAC, is used. The MAC is somewhat more complex to calculate, because most wings vary in chord over the span, growing narrower towards the outer tips. This means that more lift is generated on the wider inner portions, and the MAC moves the point to measure the chord to take this into account.

## Standard mean chord

Standard mean chord (SMC) is defined as wing area divided by wing span:[3][citation needed]

${\displaystyle {\mbox{SMC}}={\frac {S}{b}},}$

where S is the wing area and b is the span of the wing. Thus, the SMC is the chord of a rectangular wing with the same area and span as those of the given wing. This is a purely geometric figure and is rarely used in aerodynamics.

## Mean aerodynamic chord

Mean aerodynamic chord (MAC) is defined as:[4]

${\displaystyle {\mbox{MAC}}={\frac {2}{S}}}$${\displaystyle \int _{0}^{\frac {b}{2}}c(y)^{2}dy,}$

where y is the coordinate along the wing span and c is the chord at the coordinate y. Other terms are as for SMC.

The MAC is a two-dimensional representation of the whole wing. The pressure distribution over the entire wing can be reduced to a single lift force on and a moment around the aerodynamic center of the MAC. Therefore, not only the length but also the position of MAC is often important. In particular, the position of center of gravity (CG) of an aircraft is usually measured relative to the MAC, as the percentage of the distance from the leading edge of MAC to CG with respect to MAC itself.

Note that the figure to the right implies that the MAC occurs at a point where leading or trailing edge sweep changes. In general, this is not the case. Any shape other than a simple trapezoid requires evaluation of the above integral.

The ratio of the length (or span) of a rectangular-planform wing to its chord is known as the aspect ratio, an important indicator of the lift-induced drag the wing will create.[5] (For wings with planforms that are not rectangular, the aspect ratio is calculated as the square of the span divided by the wing planform area.) Wings with higher aspect ratios will have less induced drag than wings with lower aspect ratios. Induced drag is most significant at low airspeeds. This is why gliders have long slender wings.

## Tapered wing

Knowing the area (Sw), taper ratio (${\displaystyle \lambda }$) and the span (b) of the wing, the chord at any position on the span can be calculated by the formula:[6]

${\displaystyle c(y)={\frac {2\,S_{w}}{(1+\lambda )b}}\left[1-{\frac {1-\lambda }{b}}|y|\right],}$

where

${\displaystyle \lambda ={\frac {C_{\rm {Tip}}}{C_{\rm {Root}}}}}$

NOTE: This formula only works if y=0 is the port wingtip and y=b is the starboard wingtip. Typically, y=0 represents the midspan location.

Note 2: The formula as presented does not work regardless of whether one uses y = 0 -> port tip or not, and the note is not consistent with the use of the absolute value of y in the formula. The formula should read

${\displaystyle c(y)={\frac {2\,S_{w}}{(1+\lambda )b}}\left[1-{\frac {1-\lambda }{b}}|2y|\right],}$

## References

1. ^ L. J. Clancy (1975), Aerodynamics, Section 5.2, Pitman Publishing Limited, London. ISBN 0-273-01120-0
2. ^ a b Houghton, E. L.; Carpenter, P.W. (2003). Butterworth Heinmann, ed. Aerodynamics for Engineering Students (5th ed.). ISBN 0-7506-5111-3. p.18
3. ^ V., Cook, M. (2013). Flight dynamics principles : a linear systems approach to aircraft stability and control (3rd ed.). Waltham, MA: Butterworth-Heinemann. ISBN 9780080982427. OCLC 818173505.
4. ^ Abbott, I.H., and Von Doenhoff, A.E. (1959), Theory of Wing Sections, Section 1.4 (page 27), Dover Publications Inc., New York, Standard Book Number 486-60586-8
5. ^ Kermode, A.C. (1972), Mechanics of Flight, Chapter 3, (p.103, eighth edition), Pitman Publishing Limited, London ISBN 0-273-31623-0
6. ^ Ruggeri, M.C., (2009), Aerodinámica Teórica, Apuntes de la materia, UTN-FRH, Haedo, Buenos Aires


text:
### - Art Gallery -

In mathematics, an automorphic L-function is a function L(s,π,r) of a complex variable s, associated to an automorphic representation π of a reductive group G over a global field and a finite-dimensional complex representation r of the Langlands dual group LG of G, generalizing the Dirichlet L-series of a Dirichlet character and the Mellin transform of a modular form. They were introduced by Langlands (1967, 1970, 1971).

Borel (1979) and Arthur & Gelbart (1991) gave surveys of automorphic L-functions.
Properties

Automorphic L-functions should have the following properties (which have been proved in some cases but are still conjectural in other cases).

The L-function $${\displaystyle L(s,\pi ,r)}$$ should be a product over the places v of F of local L functions.

$${\displaystyle L(s,\pi ,r)=\Pi L(s,\pi _{v},r_{v})}$$

Here the automorphic representation $${\displaystyle \pi =\otimes \pi _{v}}$$ is a tensor product of the representations $$\pi _{v}$$ of local groups.

The L-function is expected to have an analytic continuation as a meromorphic function of all complex s, and satisfy a functional equation

$${\displaystyle L(s,\pi ,r)=\epsilon (s,\pi ,r)L(1-s,\pi ,r^{\lor })}$$

where the factor $${\displaystyle \epsilon (s,\pi ,r)}$$ is a product of "local constants"

$${\displaystyle \epsilon (s,\pi ,r)=\Pi \epsilon (s,\pi _{v},r_{v},\psi _{v})}$$

almost all of which are 1.
General linear groups

Godement & Jacquet (1972) constructed the automorphic L-functions for general linear groups with r the standard representation (so-called standard L-functions) and verified analytic continuation and the functional equation, by using a generalization of the method in Tate's thesis. Ubiquitous in the Langlands Program are Rankin-Selberg products of representations of GL(m) and GL(n). The resulting Rankin-Selberg L-functions satisfy a number of analytic properties, their functional equation being first proved via the Langlands–Shahidi method.

In general, the Langlands functoriality conjectures imply that automorphic L-functions of a connected reductive group are equal to products of automorphic L-functions of general linear groups. A proof of Langlands functoriality would also lead towards a thorough understanding of the analytic properties of automorphic L-functions.

References
Arthur, James; Gelbart, Stephen (1991), "Lectures on automorphic L-functions", in Coates, John; Taylor, M. J. (eds.), L-functions and arithmetic (Durham, 1989) (PDF), London Math. Soc. Lecture Note Ser., 153, Cambridge University Press, pp. 1–59, doi:10.1017/CBO9780511526053.003, ISBN 978-0-521-38619-7, MR 1110389
Borel, Armand (1979), "Automorphic L-functions", in Borel, Armand; Casselman, W. (eds.), Automorphic forms, representations and L-functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 2, Proc. Sympos. Pure Math., XXXIII, Providence, R.I.: American Mathematical Society, pp. 27–61, doi:10.1090/pspum/033.2/546608, ISBN 978-0-8218-1437-6, MR 0546608
Cogdell, James W.; Kim, Henry H.; Murty, Maruti Ram (2004), Lectures on automorphic L-functions, Fields Institute Monographs, 20, Providence, R.I.: American Mathematical Society, ISBN 978-0-8218-3516-6, MR 2071722
Gelbart, Stephen; Piatetski-Shapiro, Ilya; Rallis, Stephen (1987), Explicit constructions of automorphic L-functions, Lecture Notes in Mathematics, 1254, Berlin, New York: Springer-Verlag, doi:10.1007/BFb0078125, ISBN 978-3-540-17848-4, MR 0892097
Godement, Roger; Jacquet, Hervé (1972), Zeta functions of simple algebras, Lecture Notes in Mathematics, 260, Berlin, New York: Springer-Verlag, doi:10.1007/BFb0070263, ISBN 978-3-540-05797-0, MR 0342495
Jacquet, H.; Piatetski-Shapiro, I. I.; Shalika, J. A. (1983), "Rankin-Selberg Convolutions", Amer. J. Math., 105: 367–464, doi:10.2307/2374264
Langlands, Robert (1967), Letter to Prof. Weil
Langlands, R. P. (1970), "Problems in the theory of automorphic forms", Lectures in modern analysis and applications, III, Lecture Notes in Math, 170, Berlin, New York: Springer-Verlag, pp. 18–61, doi:10.1007/BFb0079065, ISBN 978-3-540-05284-5, MR 0302614
Langlands, Robert P. (1971) [1967], Euler products, Yale University Press, ISBN 978-0-300-01395-5, MR 0419366
Shahidi, F. (1981), "On certain "L"-functions", Amer. J. Math., 103: 297–355, doi:10.2307/2374219


text:
# What happens to Trisha's Brook? What doe sit turn into? Why Does it upset her?

This is the question in my stidy guide


text:
Euler Number

Written by Jerry Ratzlaff on . Posted in Dimensionless Numbers

Euler Number is a dimensionless number about the relation between the pressure acting on a volume of liquid or gas and the inertia forces.

FORMULA

$$Eu = \frac{\triangle p}{\rho v^2}$$

Where:

$$Eu$$ = Euler Number

$$\triangle p$$ = pressure differential

$$\rho$$ (Greek symbol rho) = density

$$v$$ = velocity

Solve for:

$$\triangle p = Eu \rho v^2$$

$$p = \frac{\triangle p}{Eu v^2}$$

$$v = \sqrt { \frac{\triangle p}{Eu \rho} }$$


text:
Go back

Build an On-Chain Puzzle Game on Celo

Introduction

In this tutorial, we will be building a puzzle game where the solution to each game will be hashed and stored on the blockchain. This tutorial is designed to teach you about Solidity development, testing, and deployment, as well as how hashing works. Let's get started!

Prerequisites

To follow this tutorial, it is recommended that you have a basic understanding of Solidity and JavaScript. If you are already familiar with the Hardhat, that will be helpful, but if not, don't worry—we will cover a few things about Hardhat in this tutorial as well. Having some familiarity with these technologies will make it easier for you to understand and follow along with the tutorial, but even if you are new to Solidity and JavaScript, you should still be able to follow along and learn a lot from this tutorial.

Requirements

To follow this tutorial, you will need to have the latest version of Node.js installed on your machine. In this tutorial, we will be using Node.js to run and test our Solidity code, so it is important that you have it installed on your machine. To check if you have installed Node.js installed on your machine, you can run following command in terminal:

node -v

If you don't have Node.js installed, you can download it from the official website and follow the instructions provided there to install it on your machine. Once you have Node.js installed, you will be ready to follow along with this tutorial.

How the Puzzle Game Works

In our puzzle game, there will be two types of people: creators and players. Creators will be responsible for creating puzzles, while players will try to solve the puzzles that have been created. When a creator creates a puzzle, they will provide a question or hint for the puzzle, as well as the hashed answer, which will be stored on-chain. To hash the answer, the creator will use a hashing function on their own machine, and then push the hashed answer to the smart contract. When a player wants to solve a puzzle, they can submit their guess to the smart contract, which will then hash the guess and compare it to the stored hashed answer for that puzzle. If the hashes match, the player's guess is correct, and they have successfully solved the puzzle.

Understanding Hashing and Its Role in the Puzzle Game

Hashing is a process of generating a fixed-size output (called a hash or hash value) from a variable-size input (such as a message or a file). The input data is processed through a mathematical algorithm (called a hash function) that produces a unique, fixed-size output. The output of a hash function is often referred to as a "digest" or a "fingerprint" of the input data, as it is unique to that specific input and can be used to identify it.

In our puzzle game, we will be using hashing to store the solutions to the puzzles on the blockchain. When a creator creates a puzzle, they will provide the question or hint for the puzzle, as well as the answer. The answer will be hashed using a function called keccak256 and the resulting hash will be stored on the blockchain along with the question or hint.

When a player wants to solve the puzzle, they can submit their guess to the smart contract, which will then hash the guess and compare it to the hashed answer stored on the blockchain. If the hashes match, the player's guess is correct and they have successfully solved the puzzle.

Hashing is useful in this context because it allows us to store the solution to the puzzle on the blockchain in a secure and tamper-proof way. Since the hash function is deterministic and collision-resistant, it is very difficult for an attacker to forge or manipulate the solution without being detected. This ensures that the puzzle game is fair and transparent for all players.

Setting Up the Hardhat Project

To set up a new Hardhat project for this tutorial, you will need to create a folder called "Puzzle" and open it in a code editor such as Visual Studio Code. Once you have the folder open, you can use the terminal to run the following command:

npx hardhat .

This command will set up a new hardhat project in the current directory (the "." indicates the current directory). It will create the necessary files and directories for your project, such as a package.json file and a contracts directory for storing your Solidity code.

To check the test coverage of our smart contract, we will install the solidity-coverage plugin. To install the plugin, run the following command in the terminal:

npm install --save-dev solidity-coverage

To use the solidity-coverage plugin, we will need to modify the hardhat.config.js file by adding the following line:

require('solidity-covergae')

Once the hardhat project has been set up, you will need to remove all the files from the contracts and test directories. Then, you can create a new Solidity file called Puzzle.sol in the contracts directory, and a new test file called puzzle.test.js in the test directory. These files will be used to develop and test your puzzle game smart contract. After creating these files, you should be ready to start coding your smart contract.

Writing the Puzzle Game Smart Contract

To begin writing the smart contract for the puzzle game, we will open Puzzle.sol and write our code in it. First, we will specify the license under which the code is released and the version of Solidity that the code is compatible with using the "pragma" directive. Then, we will define an empty smart contract called Puzzle, where we will implement all of the functions for our game.

// SPDX-License-Identifier: MITpragma solidity ^0.8.17;contract Puzzle {}

In order to store all of the details of each puzzle in the smart contract, we will create a mapping called puzzles that will store all of the puzzles. To store the details of a puzzle, we will use a struct called puzzle that contains three fields: clue, answer, and solved. The puzzles mapping will have a key of type uint256 that represents the index of a puzzle, and a value of type puzzle that represents the details of the puzzle. We will use a variable called puzzleCounter to track the index of each new puzzle that is added to the puzzles mapping. This allows us to efficiently store and retrieve the details of each puzzle in the smart contract.

    struct puzzle {        bytes32 clue;        bytes32 answer;        bool solved;    }    mapping(uint256 => puzzle) public puzzles;    uint256 public puzzleCounter;

In order to keep track of important events in the puzzle game, we will define two events in our contract: newPuzzle and solved. The newPuzzle event will be emitted whenever a new puzzle is created, and it will contain the address of the creator and the index of the new puzzle as arguments. The solved event will be emitted whenever a puzzle is solved, and it will contain the address of the solver and the index of the solved puzzle as arguments. These events can be used to trigger external actions or to provide information about the state of the game to interested parties.

    event newPuzzle( address creator, uint256 index);    event solved( address solver, uint256 index);

To create a new puzzle, we will write a function called createPuzzle with public visibility. The function will take two arguments: clue and answer, both of which are of type bytes32. The clue argument will represent the clue for the puzzle, and the answer argument will represent the hashed answer for the puzzle.

Inside the function, we will access the puzzles mapping using the current value of puzzleCounter as the key, and assign it a new puzzle using the puzzle struct, passing in the clue and answer arguments from the function parameters and setting the solved status to false to indicate that the puzzle has not yet been solved.

After creating the puzzle, we will emit the newPuzzle event, passing in the address of the creator and the index of the new puzzle as arguments. Finally, we will increment the puzzleCounter variable to keep track of the number of puzzles that have been created and to use it as the index for the next puzzle.

    function createPuzzle(bytes32 clue, bytes32 answer) public  {        puzzles[puzzleCounter] = puzzle(clue, answer, false);        emit newPuzzle(msg.sender, puzzleCounter);        puzzleCounter++;    }

In order to allow players to solve a particular puzzle, we will write a function called solvePuzzle with public visibility. The function will take two arguments: index and guess, where index is of type uint256 and represents the index of the puzzle, and guess is of type string with a storage location of calldata and represents the guess submitted by the player.

Inside the function, we will create a variable called _puzzle of type puzzle with a storage location of storage, and assign it the value of the puzzle at the given index in the puzzles mapping. We will use the require function to check if the puzzle has already been solved by checking if the solved flag is false. If the puzzle has already been solved, the function will throw an error with the message ALREADY_SOLVED.

To check if the submitted answer is the same as the stored answer, we will first pass the guess argument to the abi.encodePacked function to format it. We will then use the keccak256 function to hash the output of the abi.encodePacked function. This way, we can compare the hashed guess with the stored answer. If the hashed guess is the same as the stored answer, we will set the solved flag to true, emit the solved event with the address of the solver and the index of the puzzle as arguments, and return the string RIGHT_ANSWER. If the hashed guess is not the same as the stored answer, we will simply return the string WRONG_ANSWER.

    function solvePuzzle(        uint256 index,        string calldata guess    ) public returns (string memory) {        puzzle storage _puzzle = puzzles[index];        require(_puzzle.solved != true, "ALREADY_SOLVED");        if (_puzzle.answer == keccak256(abi.encodePacked(guess))) {            _puzzle.solved = true;            emit solved(msg.sender, index);            return "RIGHT_ANSWER";        } else {            return "WRONG_ANSWER";        }    }

Testing the Puzzle Smart Contract

To test the smart contract, we need to import certain functions and libraries that will allow us to interact with the contract and perform various checks. At the top of the file puzzle.test.js, we can start by importing the following:

1. loadFixture from @nomicfoundation/hardhat-network-helpers: This function allows us to easily load and deploy our contract during the testing process.
2. expect from chai: This function provides a way to make assertions about the contract's behavior and state.
3. utils from ethers: This library provides utility functions for working with Ethereum data types, such as converting between hexadecimal and string values.
4. ethers from hardhat: This library provides functions for interacting with the contract and the Ethereum network during the testing process.

After importing these libraries and functions, we can begin writing our test cases.

const {    loadFixture} = require("@nomicfoundation/hardhat-network-helpers");const { expect } = require("chai");const { utils } = require("ethers");const { ethers } = require("hardhat");

To set up the tests for the Puzzle contract using Hardhat, we will first create a describe block with the name "Puzzle" and define a function called contractFixture within it. The contractFixture function will be responsible for setting up the test environment and returning the necessary objects to the test cases.

Within the contractFixture function, we will use the ethers object from the hardhat package to get the signer accounts and the contract factory for the Puzzle contract.

Next, we will use the contract factory to deploy an instance of the Puzzle contract and wait for it to be deployed. Finally, we will return an object containing the signer accounts and the deployed contract instance to the test cases.

describe('Puzzle', () => {    async function contractFixture() {        const accounts = await ethers.getSigners();        const PUZZLE = await ethers.getContractFactory("Puzzle");        const puzzle = await PUZZLE.deploy();        await puzzle.deployed();        return { accounts, puzzle };    }    //--------------- All tests will go below this ------------------------------    //------------------End of the tests ----------------------------------------})

To test the smart contract in hardhat, we will create a describe block called "Puzzle" and inside it, we will use the it function to define individual tests. Each test will start by calling the loadFixture function from the @nomic-foundation/hardhat-network-helpers library. This function will setup the contract and accounts for the test. We can then use the returned object to interact with the contract and make assertions about the expected behavior. To make assertions, we will use the expect function from the chai library. After setting up the conditions for the test, we will check if we get the expected output by calling various methods on the contract and making assertions about their return values or emitted events.

For our first test, we will verify that the contract was deployed correctly by checking if the puzzleCounter variable has been initialized to 0.

    it("should be deployed correctly", async () => {        const { puzzle } = await loadFixture(contractFixture);        expect(await puzzle.puzzleCounter()).to.be.equal(0);    })

In the second test, we will check the functionality of the createPuzzle function. To do this, we will create a new puzzle and check if the newPuzzle event is emitted and the puzzleCounter increases by 1. We will also check if the values for the puzzle are correctly stored in the puzzles mapping.

    it("should create a puzzle", async () => {        const { accounts, puzzle } = await loadFixture(contractFixture);        const answer = ethers.utils.id("sun");        const clue = ethers.utils.formatBytes32String("I wake up everyone, who am I")        expect(await puzzle.createPuzzle(clue, answer)).to.emit(puzzle, "newPuzzle").withArgs(${accounts[0].address}, 0) expect(await puzzle.puzzleCounter()).to.be.equal(1); const _puzzle = await puzzle.puzzles(0); expect(_puzzle.clue).to.be.equal(clue); expect(_puzzle.solved).to.be.false; }) The third test will test the functionality of the solvePuzzle function. This test will verify that the solved event is emitted with the correct arguments and that the solved flag for the puzzle is set to true after it is solved.  it("should quzzle get solved", async () => { const { accounts, puzzle } = await loadFixture(contractFixture); const answer = ethers.utils.id("sun"); const clue = ethers.utils.formatBytes32String("I wake up everyone, who am I") await puzzle.createPuzzle(clue, answer); const result = await puzzle.solvePuzzle(0, 'sun'); expect(result).to.be.emit(puzzle, "solved").withArgs(${accounts[0].address}, 0);        const _puzzle = await puzzle.puzzles(0);        expect(_puzzle.solved).to.be.true;        await expect(puzzle.solvePuzzle(0, 'sun')).to.be.revertedWith("ALREADY_SOLVED");    })

The last test will check that the solvePuzzle function correctly reverts when an incorrect answer is submitted.

    it("should revert if wrong answer", async () => {        const { puzzle } = await loadFixture(contractFixture);        const answer = ethers.utils.id("sun");        const clue = ethers.utils.formatBytes32String("I wake up everyone, who am I")        await puzzle.createPuzzle(clue, answer);        expect(await puzzle.solvePuzzle(0, 'dogs')).to.be.revertedWith('WRONG_ANSWER');    })

To run the tests in hardhat, you have run the following command in the terminal to run the tests:

npx hardhat test

The test results should be displayed in the terminal should look similar to following:

Coverage of Smart Contract Tests

To check the coverage of our smart contract using the solidity-coverage plugin, run following command in terminal:

npx hardhat coverage

You should see report line this in the terminal:

This command will run all of the tests in your project and generate a coverage report that shows which lines of code were covered by the tests. The report will also be saved in the coverage folder of your project and can be viewed by opening the index.html file in a web browser.

Conclusion

In this tutorial, we learned about hashing and how it can be used in smart contracts. We implemented this concept using the keccak256 function in our puzzle smart contract. We then wrote tests to ensure that the contract was functioning correctly. Overall, this tutorial provided a great introduction to hashing and its practical application in Solidity.

Next Steps

In this tutorial, we not only learned about the concept of hashing in general, but we also implemented it using the keccak256 function in our puzzle game smart contract. However, there are other ways to implement hashing in smart contracts, such as using the sha256 function or creating a custom hashing function. It would be interesting to explore these other options and compare their benefits and drawbacks.

Additionally, although we have tested our puzzle game contract and achieved 100% test coverage, there may be other tests that could be added to further improve the robustness of our tests. Some ideas could include testing edge cases, such as trying to solve a puzzle with a very long or very short answer, or testing the contract's behavior when there are a large number of puzzles in the mapping. By adding these additional tests, we can increase our confidence in the contract's correctness and ensure that it is working as expected.


text:
# Why output of dvipdfmx looks better than output of pdftex?

When viewed on screen, look of document processed via dvipdfmx is better, than that of produced via pdftex.

This happens with absolutely any document, even with plain format.

I tried with these viewers:

atril xpdf evince qpdfview mupdf okular gv zathura

Each of them produces different screen output on documents, obtained via dvipdfmx (or dvips+ps2pdf) and pdftex.

As is shown on the pictures, in pdf mode the font color is grey and links color is blurry, in comparison with dvi mode.

$pdffonts dvi.pdf name type encoding emb sub uni object ID ------------------------------------ ----------------- ---------------- --- --- --- --------- FESNVM+CMSY10 Type 1C Builtin yes yes yes 4 0 XZXPQO+CMR10 Type 1C Builtin yes yes no 5 0 CSTOKE+CMR8 Type 1C Builtin yes yes no 6 0 LXXLBC+CMBX10 Type 1C Builtin yes yes no 7 0 LIFJYF+CMTI10 Type 1C Builtin yes yes no 11 0  $ pdffonts pdf.pdf
name                                 type              encoding         emb sub uni object ID
------------------------------------ ----------------- ---------------- --- --- --- ---------
IGWWSQ+CMSY10                        Type 1            Builtin          yes yes no       8  0
GVEKFU+CMR10                         Type 1            Builtin          yes yes no       9  0
JBEWTB+CMR8                          Type 1            Builtin          yes yes no      10  0
RRATMA+CMBX10                        Type 1            Builtin          yes yes no      11  0
YOSONK+CMTI10                        Type 1            Builtin          yes yes no      14  0

In one of the topics it was proposed to convert Type1 fonts to Type1C fonts via ps2pdf, but the problem is that ps2pdf converts fonts to raster images while creating pdf, thus this is not a proper way. Moreover, although the converted document is better, it is still a little blurry in comparison with the one, obtained via dvipdfmx.

I will accept if this is not possible to correct pdf viewer or pdftex, but at least why do not I get the same look of the document from pdftex and dvipdfmx?

NOTE the answer should not assume changing fonts - default Computer Modern must be used.

UPDATE

This is very strange: compile the following code with pdftex and tex+dvipdfmx - the colors are not the same at all! Maybe this is the key for understanding this mystery?

1) code for dvipdfmx:

\special{color push rgb 0 0 1}
\vrule width 5cm height 5cm
\special{color pop}
\bye

2) code for pdftex:

\input pdfcolor
\Blue
\vrule width 5cm height 5cm
\bye

And the same for \Black !!!

• What font info is reported for the two docs? – Joseph Wright Jan 14 '16 at 7:02
• using \input color \color{blue} rather than \input pdfcolor \Blue produces correct color. – user4686 Jan 14 '16 at 9:51
• In LaTeX you can use \documentclass{article} \usepackage[rgb]{xcolor} \begin{document} \color[cmyk]{1,1,0,0} \rule {5cm}{5cm} \end{document} to produce the same output as with \color{blue}. The rgb option to xcolor tells it to convert the color to rgb-model. I believe this means the pdf is then created with something akin to \pdfliteral {0 0 1 rg 0 0 1 RG} (apart from using the pdf color stack). I don't know why using a \pdfliteral color specification in cmyk model seems to result in a very different color space at the time of pdf rendering. – user4686 Jan 14 '16 at 17:46
• Perhaps this answer will prove useful ? what I understand is that the \Blue in pdfcolor.tex is in an abstract cmyk model, whose naive conversion to rgb model does correspond to R0 G0 B255, this is for example what xcolor would do if asked to do that conversion cmyk->rgb; but if one lets the cmyk color spec inscribed as is inside the pdf it will end up rendered very differently than the supposedly equivalent rgb spec. Other (non-linear ?) conversions are done PDF spec. – user4686 Jan 14 '16 at 18:00
• As I said first, you are comparing different colors. Please try %tex + dvipdfmx \input color \definecolor{myblue}{rgb}{0,0,1} \definecolor{Myblue}{cmyk}{1,1,0,0} \color{myblue} \vrule width 5cm height 5cm \color{Myblue} \hskip5mm\vrule width 5cm height 5cm \bye – Akira Kakuto Jan 15 '16 at 5:12

There were actually two clashing problems, which in the beginning I thought was one problem - and as such messed up some points in my question.

In light of newly revealed facts, I would like reclassify the above mentioned viewers into two categories.

a) the following viewers differ only with respect to color model (rgb vs. cmyk)

gv mupdf

b) the following viewers differ with respect to color model and with respect to fonts handling (Type1 vs. Type1C)

qpdfview evince okular xpdf atril zathura

1) Answer for the problem with color model:

use \pdfliteral{0 0 1 rg} and \pdfliteral{0 0 0 rg} for blue and black correspondingly

2) Answer for the problem with greyish look: convert the document with ps2pdf (it does not produce raster fonts, as I mistakenly wrote in my question)

ps2pdf -dPDFSETTINGS=/prepress -dAutoRotatePages=/None -dEmbedAllFonts=true -dSubsetFonts=true orig.pdf new.pdf

Now when I compare output of ps2pdf with output of dvipdfmx there are any differences.

The only thing left to do is to ensure that this ps2pdf command works without loss of information, i.e., how to convert back to Type1 fonts and compare hash sums with the original?

Summary: Apart from the problem with color model, I presume this to be a bug of pdf viewer and/or pdf library in handling Type1 fonts vs. Type1C fonts, because output of gv and mupdf looks absolutely the same before and after processing with ps2pdf, as it should be.