## linear algebra – Implement statistics for the length of continued fractions of a result on MATHEMATICA

After successfully generating the reduced fraction of two coprime in interval (0,1) with the following;

``````l={};
For( b=1, b<20,b++For(a= 1, a < b, a++, AppendTo(l,Simplify(a/b))));
L=DeleteDuplicates(l);
L
``````

Now, I want to implement the statistics for the length of the corresponding continued fractions, But I have no idea how to go about this using MATHEMATICA

I got a little help here if I implement this codes
`Map(Length @* ContinuedFraction, L)` `L = FareySequence(19)((2 ;; -2))` which seems equivalent and shorter.

But I will like to know the kind of continued fraction expansion used since the lengths of the continued fractions of 1/2 and 2/3, as I understand, are both equal to 2

## mysqldump – Wait state: Statistics

We have 200 databases in one instance of MariaDB with a total of 370.000 tables, since we upgraded to MariaDB 10.5.11, we see most of time passed by mysql is in wait states: Statistics.

According MariaDB, it should be a brief state:
Calculating statistics as part of deciding on a query execution plan. Usually a brief state unless the server is disk-bound.
But we don’t see particular high usage of the disk.

Trying to help this situation, we generated all Engine-Independent Table Statistics by analyzing all tables(370.000), our use_stat_tables is set to preferably_for_queries to use those stats instead of innodb one’s. No real improvement.

We also tried to switch optimizer_search_depth to 0, no improvement either.

And another consequence of this huge time passed calculating statistics, it increased by a lot our daily backup time with mysqldump of the 200 databases (before 30 min, now around 5 hours!) which crashes the server most of time, because the memory increased after each database backup, until the server swap.

Some more details, after 11 hours running

With mariadb 10.1 – 2 weeks

With MariaDB 10.5 – 2 weeks

We just received some advice to increase the open-files-limit, we still have to do it in production, and then raise table_open_cache (for now it’s table_open_cache = 20000, table_definition-cache = 40000)
Now max open files for mariadb is set to 32768:

``````cat /proc/\$(pidof mariadbd)/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 160172 160172 processes
Max open files 32768 32768 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 160172 160172 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
``````

We are running out of ideas.
Any help will be really appreciated.

Here is the output of ‘show global status’

``````| Aborted_clients                                        | 613                                              |
| Aborted_connects                                       | 1                                                |
| Aria_pagecache_blocks_not_flushed                      | 0                                                |
| Aria_pagecache_blocks_unused                           | 4                                                |
| Aria_pagecache_blocks_used                             | 15647                                            |
| Aria_pagecache_write_requests                          | 1085362                                          |
| Aria_pagecache_writes                                  | 162904                                           |
| Aria_transaction_log_syncs                             | 108                                              |
| Binlog_commits                                         | 884378                                           |
| Binlog_group_commits                                   | 884287                                           |
| Binlog_group_commit_trigger_count                      | 0                                                |
| Binlog_group_commit_trigger_lock_wait                  | 0                                                |
| Binlog_group_commit_trigger_timeout                    | 0                                                |
| Binlog_snapshot_file                                   | mysql-bin.000038                                 |
| Binlog_snapshot_position                               | 552835703                                        |
| Binlog_bytes_written                                   | 1626601753                                       |
| Binlog_cache_disk_use                                  | 4811                                             |
| Binlog_cache_use                                       | 882274                                           |
| Binlog_stmt_cache_disk_use                             | 18                                               |
| Binlog_stmt_cache_use                                  | 2104                                             |
| Busy_time                                              | 0.000000                                         |
| Bytes_sent                                             | 80380675660                                      |
| Compression                                            | OFF                                                                                           |
| Connections                                            | 143465                                           |
| Cpu_time                                               | 0.000000                                         |
| Created_tmp_disk_tables                                | 43263                                            |
| Created_tmp_files                                      | 4545                                             |
| Created_tmp_tables                                     | 961351                                                                                         |
| Delete_scan                                            | 2230                                             |
| Empty_queries                                          | 19369816
| Handler_commit                                         | 42120185                                         |
| Handler_delete                                         | 451478                                           |
| Handler_discover                                       | 80                                               |
| Handler_external_lock                                  | 0                                                |
| Handler_icp_attempts                                   | 666402417                                        |
| Handler_icp_match                                      | 666047991                                        |
| Handler_mrr_init                                       | 0                                                |
| Handler_mrr_key_refills                                | 0                                                |
| Handler_mrr_rowid_refills                              | 0                                                |
| Handler_prepare                                        | 2037194                                          |
| Handler_rollback                                       | 13                                               |
| Handler_savepoint                                      | 7376                                             |
| Handler_savepoint_rollback                             | 0                                                |
| Handler_tmp_delete                                     | 0                                                |
| Handler_tmp_update                                     | 3760231                                          |
| Handler_tmp_write                                      | 108494398                                        |
| Handler_update                                         | 723675                                           |
| Handler_write                                          | 468014                                           |
| Innodb_background_log_sync                             | 39208                                            |
| Innodb_buffer_pool_dump_status                         |                                                  |
| Innodb_buffer_pool_resize_status                       |                                                  |
| Innodb_buffer_pool_pages_data                          | 840319                                           |
| Innodb_buffer_pool_bytes_data                          | 13767786496                                      |
| Innodb_buffer_pool_pages_dirty                         | 45657                                            |
| Innodb_buffer_pool_bytes_dirty                         | 748044288                                        |
| Innodb_buffer_pool_pages_flushed                       | 206518                                           |
| Innodb_buffer_pool_pages_free                          | 321041                                           |
| Innodb_buffer_pool_pages_misc                          | 0                                                |
| Innodb_buffer_pool_pages_old                           | 310176                                           |
| Innodb_buffer_pool_pages_total                         | 1161360                                          |
| Innodb_buffer_pool_pages_lru_flushed                   | 0                                                |
| Innodb_buffer_pool_wait_free                           | 0                                                |
| Innodb_buffer_pool_write_requests                      | 12861326                                         |
| Innodb_checkpoint_age                                  | 406077659                                        |
| Innodb_checkpoint_max_age                              | 434155992                                        |
| Innodb_data_fsyncs                                     | 55314                                            |
| Innodb_data_pending_fsyncs                             | 0                                                |
| Innodb_data_pending_writes                             | 0                                                |
| Innodb_data_writes                                     | 1116053                                          |
| Innodb_data_written                                    | 2917482773                                       |
| Innodb_dblwr_pages_written                             | 131349                                           |
| Innodb_dblwr_writes                                    | 1077                                             |
| Innodb_history_list_length                             | 3                                                |
| Innodb_ibuf_free_list                                  | 1436                                             |
| Innodb_ibuf_merged_delete_marks                        | 381882                                           |
| Innodb_ibuf_merged_deletes                             | 4726                                             |
| Innodb_ibuf_merged_inserts                             | 39635                                            |
| Innodb_ibuf_merges                                     | 7900                                             |
| Innodb_ibuf_segment_size                               | 1943                                             |
| Innodb_ibuf_size                                       | 506                                              |
| Innodb_log_waits                                       | 0                                                |
| Innodb_log_write_requests                              | 341852                                           |
| Innodb_log_writes                                      | 936910                                           |
| Innodb_lsn_current                                     | 69175355827                                      |
| Innodb_lsn_flushed                                     | 69175354210                                      |
| Innodb_lsn_last_checkpoint                             | 68769278168                                      |
| Innodb_max_trx_id                                      | 96623855                                         |
| Innodb_mem_dictionary                                  | 211733568                                        |
| Innodb_os_log_fsyncs                                   | 32811                                            |
| Innodb_os_log_pending_fsyncs                           | 0                                                |
| Innodb_os_log_pending_writes                           | 0                                                |
| Innodb_os_log_written                                  | 3097520128                                       |
| Innodb_page_size                                       | 16384                                            |
| Innodb_pages_created                                   | 82283                                            |
| Innodb_pages_written                                   | 178087                                           |
| Innodb_row_lock_current_waits                          | 0                                                |
| Innodb_row_lock_time                                   | 23594                                            |
| Innodb_row_lock_time_avg                               | 291                                              |
| Innodb_row_lock_time_max                               | 20972                                            |
| Innodb_row_lock_waits                                  | 81                                               |
| Innodb_rows_deleted                                    | 451067                                           |
| Innodb_rows_inserted                                   | 466012                                           |
| Innodb_rows_updated                                    | 291502                                           |
| Innodb_system_rows_deleted                             | 0                                                |
| Innodb_system_rows_inserted                            | 0                                                |
| Innodb_system_rows_updated                             | 0                                                |
| Innodb_num_open_files                                  | 16144                                            |
| Key_blocks_not_flushed                                 | 0                                                |
| Key_blocks_unused                                      | 106026                                           |
| Key_blocks_used                                        | 2073                                             |
| Key_blocks_warm                                        | 2                                                |
| Key_write_requests                                     | 25758                                            |
| Key_writes                                             | 7625                                             |
| Last_query_cost                                        | 0.000000                                         |
| Master_gtid_wait_count                                 | 0                                                |
| Master_gtid_wait_time                                  | 0                                                |
| Master_gtid_wait_timeouts                              | 0                                                |
| Max_statement_time_exceeded                            | 0                                                |
| Max_used_connections                                   | 306                                              |
| Memory_used                                            | 1232810768                                       |
| Memory_used_initial                                    | 33626248                                         |
| Not_flushed_delayed_rows                               | 0                                                |
| Open_files                                             | 238                                              |
| Open_streams                                           | 4                                                |
| Open_table_definitions                                 | 40000                                            |
| Open_tables                                            | 16144                                            |
| Opened_files                                           | 292123                                           |
| Opened_plugin_libraries                                | 0                                                |
| Opened_table_definitions                               | 80968                                            |
| Opened_tables                                          | 288311                                           |
| Opened_views                                           | 1                                                |
| Queries                                                | 41505265                                         |
| Questions                                              | 41505265                                         |
| Rows_sent                                              | 363638953                                        |
| Select_full_join                                       | 20989                                            |
| Select_full_range_join                                 | 25263                                            |
| Select_range                                           | 19735506                                         |
| Select_range_check                                     | 0                                                |
| Select_scan                                            | 3619795                                                                                       |
| Slow_queries                                           | 91                                               |
| Sort_merge_passes                                      | 875                                              |
| Sort_priority_queue_sorts                              | 210279                                           |
| Sort_range                                             | 4010080                                          |
| Sort_rows                                              | 34727539                                         |
| Sort_scan                                              | 1066118                                          |
| Subquery_cache_hit                                     | 717168                                           |
| Subquery_cache_miss                                    | 1571758                                          |
| Syncs                                                  | 3347                                             |
| Table_locks_immediate                                  | 291843                                           |
| Table_locks_waited                                     | 60                                               |
| Table_open_cache_active_instances                      | 1                                                |
| Table_open_cache_hits                                  | 46645351                                         |
| Table_open_cache_misses                                | 2666806                                          |
| Table_open_cache_overflows                             | 271551                                                                                      |
| Threads_running                                        | 1                                                | Update_scan                                            | 2604                                             |
| Uptime                                                 | 39677
``````

My.cnf

``````(server)
binlog_format=mixed
query_cache_size       = 0
query_cache_type       = 0
query_cache_limit      = 8M
max_connections           = 450
wait_timeout              = 120
interactive_timeout       = 19200
tmp_table_size            = 24M
max_heap_table_size       = 24M
max_allowed_packet = 256M
table_open_cache                        = 20000
table_definition_cache                  = 40000
open_files_limit                        = 0
innodb_lock_wait_timeout  = 50
innodb_file_per_table
innodb_buffer_pool_size   = 18G
innodb_log_buffer_size    = 256M
innodb_log_file_size      = 512M
innodb_flush_log_at_trx_commit          = 2
innodb_flush_method                     = O_DIRECT
innodb_buffer_pool_dump_at_shutdown     = ON
analyze_sample_percentage               = 0
optimizer_search_depth=                 = 0
``````

## Ola hallengren SQL Server Index and Statistics Maintenance solution

We are using the Ola hallengren SQL Server Index and Statistics Maintenance solution from the past 6 months in our production system. Script is only used for Update Statistics not Index maintenance. Job used to take about 90-120 mins to complete which was completely normal considering the database size(1.8TB). All of a sudden the job started to take about 5-6hrs to complete from the past couple of weeks. We haven’t made any changes to the system. Each Statistics used to take less than 5 secs before now they take about 60-250 secs to complete. All this happened within a couple of days, not gradually. We are using SQL Server Enterprise edition.

Has anyone experienced this kind of issue before?
Any suggestions are greatly appreciated.

Below are the parameters used in SQL job.

EXECUTE dbo.IndexOptimize
–@Databases = ‘ALL_DATABASES’,
@Databases = ‘User DB’,
@FragmentationLow = NULL,
@FragmentationMedium = NULL,
@FragmentationHigh = NULL,
@OnlyModifiedStatistics = ‘Y’,
@MAXDOP=2,
@Indexes=’ALL_INDEXES’,
@LogToTable=’Y’
–,@Indexes=’ALL_INDEXES,-%.dbo.eventhistory,-%.dbo.eventhistoryrgc’

Best Regards,
Arun

## Where can I find the Android API level statistics online?

For the longest time, Google have displayed Android API level information here: https://developer.android.com/about/dashboards/

Unfortunately when I went to check it today, I found the following message:

You can find platform version information in Android Studio’s Create New Project wizard.

I already have an existing project in Android Studio, and I’m considering whether to change the `minSdkVersion`. I don’t want to create a new project just to check the Android API level statistics.

Where can I check the Android API level statistics without creating a new Android Studio project?

## probability or statistics – Finding the error type I

let $$x_1…. x_n$$ be a random sample from a Poisson distribution with mean $$theta$$, that is

$$f(x ;theta$$)$$= theta^x$$.$$e^-theta$$/$$x!$$.

We use a test that accepts the null hypothesis if $$(1/3)≤ overline{x} ≤ (2/3)$$ and reject otherwise.

For n=9 what is the error type I?

## statistics – Anydice: Counting Successes and Rerolling Failures In Dice Pools

The issue you’re running into here is that AnyDice loves summing numbers, especially when outputing. The output you’re getting is the sum of the dice, not the number of successes (hence going up to 60). The easiest way around this, is to have the function work out the number of successes, which we can handily do by comparing our `ROLL` sequence to our `TARGET` number. Also handily, to not have to worry about actually shuffling around the sequence to get the reroll, we can simply add the new roll and subtract a 1 (to compensate for the roll that should be removed). We also know that if we don’t reroll, we have no successes. The resulting function then becomes:

``````function: reroll greatest of ROLL:s as REROLL:n if greater than TARGET:n {
if 1@ROLL >= TARGET {
result: (ROLL >= TARGET) + (REROLL >= TARGET) -1
} else {
result: 0
}
}
``````

Which we can test over a range of targets (you’ll want to limit the number in that range so it doesn’t time out):

``````loop N over {10,12,14,16,18,20} {
output (reroll greatest of 3d20 as 1d20 if greater than N) named "(N)"
}
``````

And see the transposed graph to see how the probability of a given number of sucesses vary with the target.

## statistics – Relation between moments a measured PMF under matrix multiplication?

I’m working on a physics problem where we have a measured photon energy spectrum (I’m thinking of it as a probability mass function), which is created by an energy spectrum of electrons which impact the atmosphere. The two spectra are related by a (known) matrix multiplication.

If it’s valid to think about the two spectra as PMF’s, is there a relation between the first few moments of the two distributions?

I’m asking because the matrix is very ill-conditioned, and the first few moments of the resulting spectrum is all I need to know for the physics problem. The “full problem” of finding the inverse of the ill-conditioned matrix is handled through regularization (Tikhonov, LASSO, etc.) Is it possible to pose a better conditioned problem, by seeking “less information” about the transformed distribution?

I did some searching and found this: Characteristic Function and Random Variable Transformation. I think that the result there pertains to individual bins in my measured spectrum being considered together as a vector random variable. I can, of course, get the expectation value and the variance of each bin, but what I’m looking for is a way to get the center and approximate width of the resulting distribution, without solving the full inverse problem.

Many thanks

## ra.rings and algebras – Statistics word problem

ra.rings and algebras – Statistics word problem – MathOverflow

## statistics – Cochran’s Q Alternative for non-dictonomous data

I’m trying to compare mean spending across ethnicities. The ethnicity data is ‘select all that apply’, so a subject may be counted in more than one ethnicity but not necessarily. I’m using Cochran’s Q for related proportional data, but am having trouble finding a method to compare the mean spending.

## probability or statistics – Sample discrete distributions with a good range of observed entropies

For a numerical sanity check, I need to sample random sequences of $$n$$ positive numbers adding up to 1, and having a high chance of observing both high entropy and low entropy sequences. Compute the entropy by treating sequence as a discrete probability distribution.

Ideally, the histogram of sampled sequence entropies would approach uniform distribution.

Can someone suggest a way to do this in Mathematica? Here’s a naive generation method, showing non-uniform entropy histogram, and the kind of histogram I would like to see.

``````n = 10;
s = 10000;
normalize[seq_] := seq/Total@seq;
sequences =
normalize /@ RandomVariate[UniformDistribution[], {s, n}];
entropy[seq_] := -Total[# Log[#] & /@ seq];
Histogram[entropy /@ sequences, PlotLabel -> "observed"]
Histogram[RandomVariate[UniformDistribution[], s],
PlotLabel -> "desired"]
``````