MySQL's maximum memory usage is dangerously high



  • I'm trying to understand what is not working in my configuration.

    When I run the

        [root@server ~]# ./mysqltuner.pl
     >>  MySQLTuner 1.9.8
             * Jean-Marie Renouard 
             * Major Hayden 
     >>  Bug reports, feature requests, and downloads at http://mysqltuner.pl/
     >>  Run with '--help' for additional options and output filtering
    

    [--] Skipped version check for MySQLTuner script
    [!!] Your MySQL version 10.2.43-MariaDB-log is EOL software! Upgrade soon!
    [OK] Operating on 64-bit architecture

    -------- Log file Recommendations ------------------------------------------------------------------
    [OK] Log file /var/log/mysqld.log exists
    [--] Log file: /var/log/mysqld.log(1M)
    [OK] Log file /var/log/mysqld.log is not empty
    [OK] Log file /var/log/mysqld.log is smaller than 32 Mb
    [OK] Log file /var/log/mysqld.log is readable.
    [!!] /var/log/mysqld.log contains 7295 warning(s).
    [!!] /var/log/mysqld.log contains 2416 error(s).
    [--] 68 start(s) detected in /var/log/mysqld.log
    [--] 1) 2022-04-15 15:28:49 139969276774592 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 2) 2022-04-15 15:01:58 139671548569792 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 3) 2022-04-15 15:00:37 139816143530176 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 4) 2022-04-11 15:13:19 140028901791936 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 5) 2022-03-29 2:44:09 140256877758656 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 6) 2022-03-22 2:42:21 140486199806144 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 7) 2022-02-14 2:42:44 139938060060864 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 8) 2022-02-14 2:42:41 139960106510528 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 9) 2022-02-09 2:45:31 140350442416320 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 10) 2022-02-09 2:45:28 139988696365248 [Note] /usr/sbin/mysqld: ready for connections.
    [--] 49 shutdown(s) detected in /var/log/mysqld.log
    [--] 1) 2022-04-15 15:28:49 139670594152192 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 2) 2022-04-15 15:01:57 139815146489600 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 3) 2022-04-15 14:58:06 140027817096960 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 4) 2022-04-11 15:12:31 140255785293568 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 5) 2022-03-29 2:44:09 140485092288256 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 6) 2022-03-22 2:42:20 139937820497664 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 7) 2022-02-14 2:42:44 139959094945536 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 8) 2022-02-14 2:42:40 140349468514048 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 9) 2022-02-09 2:45:31 139987684747008 [Note] /usr/sbin/mysqld: Shutdown complete
    [--] 10) 2022-02-09 2:45:27 140367091803904 [Note] /usr/sbin/mysqld: Shutdown complete

    -------- Storage Engine Statistics -----------------------------------------------------------------
    [--] Status: +Aria +CSV +InnoDB +MEMORY +MRG_MyISAM +MyISAM +PERFORMANCE_SCHEMA +SEQUENCE
    [--] Data in InnoDB tables: 3.9G (Tables: 718)
    [OK] Total fragmented tables: 0

    -------- Analysis Performance Metrics --------------------------------------------------------------
    [--] innodb_stats_on_metadata: OFF
    [OK] No stat updates during querying INFORMATION_SCHEMA.

    -------- Views Metrics -----------------------------------------------------------------------------

    -------- Triggers Metrics --------------------------------------------------------------------------

    -------- Routines Metrics --------------------------------------------------------------------------

    -------- Security Recommendations ------------------------------------------------------------------
    [OK] There are no anonymous accounts for any database users
    [OK] All database users have passwords assigned
    [!!] There is no basic password file list!

    -------- CVE Security Recommendations --------------------------------------------------------------
    [--] Skipped due to --cvefile option undefined

    -------- Performance Metrics -----------------------------------------------------------------------
    [--] Up for: 30m 9s (233K q [128.918 qps], 1K conn, TX: 1G, RX: 56M)
    [--] Reads / Writes: 93% / 7%
    [--] Binary logging is disabled
    [--] Physical Memory : 7.6G
    [--] Max MySQL memory : 45.1G
    [--] Other process memory: 0B
    [--] Total buffers: 416.0M global + 302.3M per thread (151 max threads)
    [--] P_S Max memory usage: 93M
    [--] Galera GCache Max memory usage: 0B
    [OK] Maximum reached memory usage: 3.4G (45.17% of installed RAM)
    [!!] Maximum possible memory usage: 45.1G (590.18% of installed RAM)
    [!!] Overall possible memory usage with other process exceeded memory
    [OK] Slow queries: 0% (33/233K)
    [OK] Highest usage of available connections: 6% (10/151)
    [OK] Aborted connections: 0.14% (2/1476)
    [!!] CPanel and Flex system skip-name-resolve should be on
    [OK] Query cache is disabled by default due to mutex contention on multiprocessor machines.
    [OK] Sorts requiring temporary tables: 0% (0 temp sorts / 40K sorts)
    [!!] Joins performed without indexes: 559
    [!!] Temporary tables created on disk: 87% (28K on disk / 32K total)
    [OK] Thread cache hit rate: 99% (10 created / 1K connections)
    [OK] Table cache hit rate: 99% (865 hits / 872 requests)
    [OK] table_definition_cache(2097152) is upper than number of tables(984)
    [OK] Open file limit used: 0% (38/40K)
    [OK] Table locks acquired immediately: 100% (134 immediate / 134 locks)

    -------- Performance schema ------------------------------------------------------------------------
    [--] Performance_schema is activated.
    [--] Memory used by P_S: 93.6M
    [--] Sys schema is installed.

    -------- ThreadPool Metrics ------------------------------------------------------------------------
    [--] ThreadPool stat is disabled.

    -------- MyISAM Metrics ----------------------------------------------------------------------------
    [!!] Key buffer used: 18.3% (23.4M used / 128.0M cache)
    [OK] Key buffer size / total MyISAM indexes: 128.0M/137.0K
    [OK] Read Key buffer hit rate: 96.6% (716 cached / 24 reads)

    -------- InnoDB Metrics ----------------------------------------------------------------------------
    [--] InnoDB is enabled.
    [--] InnoDB Thread Concurrency: 0
    [OK] InnoDB File per table is activated
    [!!] InnoDB buffer pool / data size: 128.0M/3.9G
    [OK] Ratio InnoDB log file size / InnoDB Buffer pool size: 14.0M * 2/128.0M should be equal to 25%
    [OK] InnoDB buffer pool instances: 1
    [--] Number of InnoDB Buffer Pool Chunk : 1 for 1 Buffer Pool Instance(s)
    [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances
    [OK] InnoDB Read buffer efficiency: 99.63% (202127202 hits/ 202873879 total)
    [!!] InnoDB Write Log efficiency: 83.74% (66789 hits/ 79762 total)
    [OK] InnoDB log waits: 0.00% (0 waits / 12973 writes)

    -------- Aria Metrics ------------------------------------------------------------------------------
    [--] Aria Storage Engine is enabled.
    [OK] Aria pagecache size / total Aria indexes: 128.0M/0B
    [OK] Aria pagecache hit rate: 98.7% (1M cached / 24K reads)

    -------- TokuDB Metrics ----------------------------------------------------------------------------
    [--] TokuDB is disabled.

    -------- XtraDB Metrics ----------------------------------------------------------------------------
    [--] XtraDB is disabled.

    -------- Galera Metrics ----------------------------------------------------------------------------
    [--] Galera is disabled.

    -------- Replication Metrics -----------------------------------------------------------------------
    [--] Galera Synchronous replication: NO
    [--] No replication slave(s) for this server.
    [--] Binlog format: MIXED
    [--] XA support enabled: ON
    [--] Semi synchronous replication Master: Not Activated
    [--] Semi synchronous replication Slave: Not Activated
    [--] This is a standalone server

    -------- Recommendations ---------------------------------------------------------------------------
    General recommendations:
    You are using n unsupported version for production environments
    Upgrade as soon as possible to a supported version !
    Check warning line(s) in /var/log/mysqld.log file
    Check error line(s) in /var/log/mysqld.log file
    MySQL was started within the last 24 hours - recommendations may be inaccurate
    Reduce your overall MySQL memory footprint for system stability
    Dedicate this server to your database for highest performance.
    name resolution is enabled due to cPanel doesn't support this disabled.
    We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found.
    See https://dev.mysql.com/doc/internals/en/join-buffer-size.html
    (specially the conclusions at the bottom of the page).
    When making adjustments, make tmp_table_size/max_heap_table_size equal
    Reduce your SELECT DISTINCT queries which have no LIMIT clause
    Variables to adjust:
    *** MySQL's maximum memory usage is dangerously high ***
    *** Add RAM before increasing MySQL buffer variables ***
    skip-name-resolve=0
    join_buffer_size (> 40.0M, or always use indexes with JOINs)
    tmp_table_size (> 16M)
    max_heap_table_size (> 16M)
    key_buffer_size (~ 24M)
    innodb_buffer_pool_size (>= 3.9G) if possible.

    I am really concerned by the "MySQL's maximum memory usage is dangerously high" message, because that means that the server can crash unexpectedly if MySQL tries to get more memory than RAM.

    Here is my my.cnf :

    [root@server ~]# cat /etc/my.cnf
    # For advice on how to change settings please see
    # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
    

    [mysqld]
    performance-schema=0

    Remove leading # and set to the amount of RAM for the most important data

    cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.

    innodb_buffer_pool_size = 128M

    Remove leading # to turn on a very important data integrity option: logging

    changes to the binary log between backups.

    log_bin

    Remove leading # to set options mainly useful for reporting servers.

    The server defaults are faster for transactions and fast SELECTs.

    Adjust sizes as needed, experiment to find the optimal values.

    join_buffer_size = 40M
    sort_buffer_size = 2M
    read_buffer_size = 2M
    read_rnd_buffer_size = 2M
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock

    Disabling symbolic-links is recommended to prevent assorted security risks

    symbolic-links=0

    log-error=/var/log/mysqld.log

    pid-file=/var/run/mysqld/mysqld.pid

    max_allowed_packet=16M
    open_files_limit=40000
    innodb_file_per_table=1

    query_cache_size=0
    query_cache_type=0
    query_cache_limit=1M
    tmp_table_size=16M
    max_heap_table_size=16M
    performance_schema=ON
    innodb_log_file_size=14M
    table_definition_cache=-1
    slow_query_log = 1
    slow-query_log_file = /var/log/mysql-slow.log
    long_query_time = 2

    I tried to change the join_buffer_size = 40M sort_buffer_size = 2M read_buffer_size = 2M read_rnd_buffer_size = 2M

    but it seems to get worse. My initial my.cnf was :

    [root@server ~]# cat /etc/my.cnf.backup
    # For advice on how to change settings please see
    # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
    

    [mysqld]
    performance-schema=0

    Remove leading # and set to the amount of RAM for the most important data

    cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.

    innodb_buffer_pool_size = 128M

    Remove leading # to turn on a very important data integrity option: logging

    changes to the binary log between backups.

    log_bin

    Remove leading # to set options mainly useful for reporting servers.

    The server defaults are faster for transactions and fast SELECTs.

    Adjust sizes as needed, experiment to find the optimal values.

    join_buffer_size = 128M

    sort_buffer_size = 2M

    read_rnd_buffer_size = 2M

    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock

    Disabling symbolic-links is recommended to prevent assorted security risks

    symbolic-links=0

    log-error=/var/log/mysqld.log

    pid-file=/var/run/mysqld/mysqld.pid

    innodb_buffer_pool_size=122683392
    max_allowed_packet=268435456
    open_files_limit=40000
    innodb_file_per_table=1

    query_cache_size=0
    query_cache_type=0
    query_cache_limit=1M
    tmp_table_size=16M
    max_heap_table_size=16M
    performance_schema=ON
    innodb_buffer_pool_size=260M
    innodb_log_file_size=14M
    table_definition_cache=-1
    slow_query_log = 1
    slow-query_log_file = /var/log/mysql-slow.log
    long_query_time = 2

    Any suggestion, or blatant error or misconfiguration that anyone could see ?



  • This is an old default:

    innodb_buffer_pool_size = 128M
    

    Raise the setting to 5G for better performance. However, this is going in the opposite direction from your Question. Offhand, I don't see any settings that are "too large".

    max_connections = 151 is OK. So, that tool must be complaining about "302.3M per thread".

    "Temporary tables created on disk" (and other things) imply that some tables are poorly indexed and/or queries poorly written. You seem to have the slowlog turned on. Use pt-query-digest to discover which query is the "worst".

    Be aware that their formula is imperfect. (There is no perfect formula.) Still, that tool found something. Let's dig deeper; provide the info asked for here: http://mysql.rjweb.org/doc.php/mysql_analysis#tuning

    As for "crashing" due to not enough RAM, setting up "swap" will cause the database to slow down instead of crashing.




Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2