MySQL eats up all the memory from the RHEL OS












0















Currently, I have MySQL server installed in the AWS RHEL EC2 instance.



The team was running heavy data processing in the DB and we found out that every now and then the DB restarted itself. After troubleshooting, we realized that the DB was actually killed by the OS! Because the MySQL is so hungry that it consumes all the memory in the OS.



The OS has 124GB RAM and 68GB is allocated to innodb_buffer_pool_size.
Attached my.cnf file



Question:



How do we prevent the OS from killing the mysqld?



    [mysqld]
# Server configuration
user=mysql
port=3306
socket=/home/mysql/server/run/mysql.sock

# File locations
datadir=/mysql/data
basedir=/home/mysql/server
lc_messages_dir=/usr/share/mysql-8.0
plugin-dir=/lib64/mysql/plugin
innodb_log_group_home_dir=/redolog/redologfiles
innodb_data_file_path=ibdata1:12M:autoextend
log-error=/home/mysql/server/logs/mysqld.log
innodb_log_file_size=2147483648
innodb_log_files_in_group=8
innodb_page_size=16384
innodb_checksum_algorithm=crc32
innodb_buffer_pool_filename=ib_buffer_pool
innodb_undo_tablespaces=2
innodb_buffer_pool_filename=ib_buffer_pool
skip-log-bin

# New server config
character-set-server=latin1
collation-server=latin1_swedish_ci
connect_timeout=31530000
net_read_timeout=31530000
net_write_timeout=31530000
innodb_buffer_pool_size=68719476736
innodb_buffer_pool_instances=64
innodb_temp_data_file_path=ibtmp1:512M:autoextend:max:512000M
innodb_max_undo_log_size=4294967296
innodb_read_io_threads=16
binlog_expire_logs_seconds=259200
#expire_logs_days=10
max_binlog_size=1073741824
sql_mode = ''
max_connections=300
log_timestamps=SYSTEM
local_infile = 1
max_digest_length = 3072
performance_schema_max_digest_length=10240
max_digest_length=10240
performance_schema_max_sql_text_length=10240
innodb_adaptive_hash_index = 0

interactive_timeout = 86400
mysqlx_interactive_timeout = 86400
mysqlx_wait_timeout = 86400
wait_timeout = 86400

general_log = OFF
general_log_file = /home/mysql/server/logs/mysqld_db.log
[client]
port=3306
socket=/home/mysql/server/run/mysql.sock

#slow_query_log_file=/home/mysql/server/logs/mysql-slow.log
#slow_query_log=ON
#general_log_file=/home/mysql/server/logs/mysql_gen_db.log
#general_log=ON








share



























    0















    Currently, I have MySQL server installed in the AWS RHEL EC2 instance.



    The team was running heavy data processing in the DB and we found out that every now and then the DB restarted itself. After troubleshooting, we realized that the DB was actually killed by the OS! Because the MySQL is so hungry that it consumes all the memory in the OS.



    The OS has 124GB RAM and 68GB is allocated to innodb_buffer_pool_size.
    Attached my.cnf file



    Question:



    How do we prevent the OS from killing the mysqld?



        [mysqld]
    # Server configuration
    user=mysql
    port=3306
    socket=/home/mysql/server/run/mysql.sock

    # File locations
    datadir=/mysql/data
    basedir=/home/mysql/server
    lc_messages_dir=/usr/share/mysql-8.0
    plugin-dir=/lib64/mysql/plugin
    innodb_log_group_home_dir=/redolog/redologfiles
    innodb_data_file_path=ibdata1:12M:autoextend
    log-error=/home/mysql/server/logs/mysqld.log
    innodb_log_file_size=2147483648
    innodb_log_files_in_group=8
    innodb_page_size=16384
    innodb_checksum_algorithm=crc32
    innodb_buffer_pool_filename=ib_buffer_pool
    innodb_undo_tablespaces=2
    innodb_buffer_pool_filename=ib_buffer_pool
    skip-log-bin

    # New server config
    character-set-server=latin1
    collation-server=latin1_swedish_ci
    connect_timeout=31530000
    net_read_timeout=31530000
    net_write_timeout=31530000
    innodb_buffer_pool_size=68719476736
    innodb_buffer_pool_instances=64
    innodb_temp_data_file_path=ibtmp1:512M:autoextend:max:512000M
    innodb_max_undo_log_size=4294967296
    innodb_read_io_threads=16
    binlog_expire_logs_seconds=259200
    #expire_logs_days=10
    max_binlog_size=1073741824
    sql_mode = ''
    max_connections=300
    log_timestamps=SYSTEM
    local_infile = 1
    max_digest_length = 3072
    performance_schema_max_digest_length=10240
    max_digest_length=10240
    performance_schema_max_sql_text_length=10240
    innodb_adaptive_hash_index = 0

    interactive_timeout = 86400
    mysqlx_interactive_timeout = 86400
    mysqlx_wait_timeout = 86400
    wait_timeout = 86400

    general_log = OFF
    general_log_file = /home/mysql/server/logs/mysqld_db.log
    [client]
    port=3306
    socket=/home/mysql/server/run/mysql.sock

    #slow_query_log_file=/home/mysql/server/logs/mysql-slow.log
    #slow_query_log=ON
    #general_log_file=/home/mysql/server/logs/mysql_gen_db.log
    #general_log=ON








    share

























      0












      0








      0








      Currently, I have MySQL server installed in the AWS RHEL EC2 instance.



      The team was running heavy data processing in the DB and we found out that every now and then the DB restarted itself. After troubleshooting, we realized that the DB was actually killed by the OS! Because the MySQL is so hungry that it consumes all the memory in the OS.



      The OS has 124GB RAM and 68GB is allocated to innodb_buffer_pool_size.
      Attached my.cnf file



      Question:



      How do we prevent the OS from killing the mysqld?



          [mysqld]
      # Server configuration
      user=mysql
      port=3306
      socket=/home/mysql/server/run/mysql.sock

      # File locations
      datadir=/mysql/data
      basedir=/home/mysql/server
      lc_messages_dir=/usr/share/mysql-8.0
      plugin-dir=/lib64/mysql/plugin
      innodb_log_group_home_dir=/redolog/redologfiles
      innodb_data_file_path=ibdata1:12M:autoextend
      log-error=/home/mysql/server/logs/mysqld.log
      innodb_log_file_size=2147483648
      innodb_log_files_in_group=8
      innodb_page_size=16384
      innodb_checksum_algorithm=crc32
      innodb_buffer_pool_filename=ib_buffer_pool
      innodb_undo_tablespaces=2
      innodb_buffer_pool_filename=ib_buffer_pool
      skip-log-bin

      # New server config
      character-set-server=latin1
      collation-server=latin1_swedish_ci
      connect_timeout=31530000
      net_read_timeout=31530000
      net_write_timeout=31530000
      innodb_buffer_pool_size=68719476736
      innodb_buffer_pool_instances=64
      innodb_temp_data_file_path=ibtmp1:512M:autoextend:max:512000M
      innodb_max_undo_log_size=4294967296
      innodb_read_io_threads=16
      binlog_expire_logs_seconds=259200
      #expire_logs_days=10
      max_binlog_size=1073741824
      sql_mode = ''
      max_connections=300
      log_timestamps=SYSTEM
      local_infile = 1
      max_digest_length = 3072
      performance_schema_max_digest_length=10240
      max_digest_length=10240
      performance_schema_max_sql_text_length=10240
      innodb_adaptive_hash_index = 0

      interactive_timeout = 86400
      mysqlx_interactive_timeout = 86400
      mysqlx_wait_timeout = 86400
      wait_timeout = 86400

      general_log = OFF
      general_log_file = /home/mysql/server/logs/mysqld_db.log
      [client]
      port=3306
      socket=/home/mysql/server/run/mysql.sock

      #slow_query_log_file=/home/mysql/server/logs/mysql-slow.log
      #slow_query_log=ON
      #general_log_file=/home/mysql/server/logs/mysql_gen_db.log
      #general_log=ON








      share














      Currently, I have MySQL server installed in the AWS RHEL EC2 instance.



      The team was running heavy data processing in the DB and we found out that every now and then the DB restarted itself. After troubleshooting, we realized that the DB was actually killed by the OS! Because the MySQL is so hungry that it consumes all the memory in the OS.



      The OS has 124GB RAM and 68GB is allocated to innodb_buffer_pool_size.
      Attached my.cnf file



      Question:



      How do we prevent the OS from killing the mysqld?



          [mysqld]
      # Server configuration
      user=mysql
      port=3306
      socket=/home/mysql/server/run/mysql.sock

      # File locations
      datadir=/mysql/data
      basedir=/home/mysql/server
      lc_messages_dir=/usr/share/mysql-8.0
      plugin-dir=/lib64/mysql/plugin
      innodb_log_group_home_dir=/redolog/redologfiles
      innodb_data_file_path=ibdata1:12M:autoextend
      log-error=/home/mysql/server/logs/mysqld.log
      innodb_log_file_size=2147483648
      innodb_log_files_in_group=8
      innodb_page_size=16384
      innodb_checksum_algorithm=crc32
      innodb_buffer_pool_filename=ib_buffer_pool
      innodb_undo_tablespaces=2
      innodb_buffer_pool_filename=ib_buffer_pool
      skip-log-bin

      # New server config
      character-set-server=latin1
      collation-server=latin1_swedish_ci
      connect_timeout=31530000
      net_read_timeout=31530000
      net_write_timeout=31530000
      innodb_buffer_pool_size=68719476736
      innodb_buffer_pool_instances=64
      innodb_temp_data_file_path=ibtmp1:512M:autoextend:max:512000M
      innodb_max_undo_log_size=4294967296
      innodb_read_io_threads=16
      binlog_expire_logs_seconds=259200
      #expire_logs_days=10
      max_binlog_size=1073741824
      sql_mode = ''
      max_connections=300
      log_timestamps=SYSTEM
      local_infile = 1
      max_digest_length = 3072
      performance_schema_max_digest_length=10240
      max_digest_length=10240
      performance_schema_max_sql_text_length=10240
      innodb_adaptive_hash_index = 0

      interactive_timeout = 86400
      mysqlx_interactive_timeout = 86400
      mysqlx_wait_timeout = 86400
      wait_timeout = 86400

      general_log = OFF
      general_log_file = /home/mysql/server/logs/mysqld_db.log
      [client]
      port=3306
      socket=/home/mysql/server/run/mysql.sock

      #slow_query_log_file=/home/mysql/server/logs/mysql-slow.log
      #slow_query_log=ON
      #general_log_file=/home/mysql/server/logs/mysql_gen_db.log
      #general_log=ON






      mysql aws redhat





      share












      share










      share



      share










      asked 8 mins ago









      JupyterousJupyterous

      12




      12






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "182"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f227931%2fmysql-eats-up-all-the-memory-from-the-rhel-os%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Database Administrators Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f227931%2fmysql-eats-up-all-the-memory-from-the-rhel-os%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          الفوسفات في المغرب

          Four equal circles intersect: What is the area of the small shaded portion and its height

          جامعة ليفربول