Server Optimization Request - Large Forum
January 9th, 2009Our forum is relatively large and running on a single dedicated server. We have limited funds so buying another server is out of the question. I am hoping we can find some optimizations that we can make so that our site will run more smoothly. We've disabled a lot of the performance hits built into vB already and switched search over to sphinx that gets indexed nightly. But we still have some massive queries that kill our database during the day. Any help would be appreciated. The answers to the questions are listed below.
1. Dedicated server.
2. Server Stats
CPU Dual AMD Operton 280 (dual core, 4 cores total)
RAM 4gb RAM
HD 2x 65GB SCSI RAID 1
OS CentOS 4.5
Apache 2.0.59
PHP 5.2.3
MySQL 4.1.22
3. vB 3.6.4
4. No innodb
5. No extra config options.
6. Top Stats
top - 12:02:52 up 119 days, 13:48, 2 users, load average: 3.15, 3.98, 3.30
Tasks: 133 total, 1 running, 132 sleeping, 0 stopped, 0 zombie
Cpu0 : 62.2% us, 6.0% sy, 0.0% ni, 31.3% id, 0.5% wa, 0.0% hi, 0.0% si
Cpu1 : 12.4% us, 2.0% sy, 0.0% ni, 83.6% id, 2.0% wa, 0.0% hi, 0.0% si
Cpu2 : 28.4% us, 2.5% sy, 0.0% ni, 64.7% id, 4.5% wa, 0.0% hi, 0.0% si
Cpu3 : 24.4% us, 2.0% sy, 0.0% ni, 73.6% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 4146500k total, 3709024k used, 437476k free, 435436k buffers
Swap: 2040244k total, 95512k used, 1944732k free, 2001960k cached
7. mySQL Config
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 384M
max_allowed_packet = 1M
table_cache = 512
sort_buffer_size = 2M
read_buffer_size = 2M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 64M
thread_cache_size = 8
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8
max_connections = 500
wait_timeout = 120
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 1024M
sort_buffer_size = 1024M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
8. mySQL Stats
+---------------------------------+------------------------------------------+
Variable_name Value
+---------------------------------+------------------------------------------+
back_log 50
basedir /
binlog_cache_size 32768
bulk_insert_buffer_size 8388608
character_set_client latin1
character_set_connection latin1
character_set_database latin1
character_set_results latin1
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection latin1_swedish_ci
collation_database latin1_swedish_ci
collation_server latin1_swedish_ci
concurrent_insert ON
connect_timeout 5
datadir /var/lib/mysql/
date_format %Y-%m-%d
datetime_format %Y-%m-%d %H:%i:%s
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
expire_logs_days 0
flush OFF
flush_time 0
ft_boolean_syntax + -><()~*:""&
ft_max_word_len 84
ft_min_word_len 4
ft_query_expansion_limit 20
ft_stopword_file (built-in)
group_concat_max_len 1024
have_archive NO
have_bdb NO
have_blackhole_engine NO
have_compress YES
have_crypt YES
have_csv NO
have_example_engine NO
have_geometry YES
have_innodb YES
have_isam NO
have_merge_engine YES
have_ndbcluster NO
have_openssl NO
have_query_cache YES
have_raid NO
have_rtree_keys YES
have_symlink YES
init_connect
init_file
init_slave
innodb_additional_mem_pool_size 1048576
innodb_autoextend_increment 8
innodb_buffer_pool_awe_mem_mb 0
innodb_buffer_pool_size 8388608
innodb_data_file_path ibdata1:10M:autoextend
innodb_data_home_dir
innodb_fast_shutdown ON
innodb_file_io_threads 4
innodb_file_per_table OFF
innodb_flush_log_at_trx_commit 1
innodb_flush_method
innodb_force_recovery 0
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_arch_dir
innodb_log_archive OFF
innodb_log_buffer_size 1048576
innodb_log_file_size 5242880
innodb_log_files_in_group 2
innodb_log_group_home_dir ./
innodb_max_dirty_pages_pct 90
innodb_max_purge_lag 0
innodb_mirrored_log_groups 1
innodb_open_files 300
innodb_table_locks ON
innodb_thread_concurrency 8
interactive_timeout 28800
join_buffer_size 131072
key_buffer_size 402653184
key_cache_age_threshold 300
key_cache_block_size 1024
key_cache_division_limit 100
language /usr/share/mysql/english/
large_files_support ON
lc_time_names en_US
license GPL
local_infile ON
locked_in_memory OFF
log OFF
log_bin OFF
log_error
log_slave_updates OFF
log_slow_queries ON
log_update OFF
log_warnings 1
long_query_time 10
low_priority_updates OFF
lower_case_file_system OFF
lower_case_table_names 0
max_allowed_packet 1047552
max_binlog_cache_size 4294967295
max_binlog_size 1073741824
max_connect_errors 10
max_connections 500
max_delayed_threads 20
max_error_count 64
max_heap_table_size 16777216
max_insert_delayed_threads 20
max_join_size 4294967295
max_length_for_sort_data 1024
max_prepared_stmt_count 16382
max_relay_log_size 0
max_seeks_for_key 4294967295
max_sort_length 1024
max_tmp_tables 32
max_user_connections 0
max_write_lock_count 4294967295
myisam_data_pointer_size 4
myisam_max_extra_sort_file_size 2147483648
myisam_max_sort_file_size 2147483647
myisam_recover_options OFF
myisam_repair_threads 1
myisam_sort_buffer_size 67108864
myisam_stats_method nulls_unequal
net_buffer_length 16384
net_read_timeout 30
net_retry_count 10
net_write_timeout 60
new OFF
old_passwords OFF
open_files_limit 4096
pid_file /var/lib/mysql/www.tribalwar.com.pid
port 3306
preload_buffer_size 32768
prepared_stmt_count 0
protocol_version 10
query_alloc_block_size 8192
query_cache_limit 1048576
query_cache_min_res_unit 4096
query_cache_size 33554432
query_cache_type ON
query_cache_wlock_invalidate OFF
query_prealloc_size 8192
range_alloc_block_size 2048
read_buffer_size 2093056
read_only OFF
read_rnd_buffer_size 8384512
relay_log_purge ON
relay_log_space_limit 0
rpl_recovery_rank 0
secure_auth OFF
server_id 1
skip_external_locking ON
skip_networking OFF
skip_show_database OFF
slave_net_timeout 3600
slave_transaction_retries 0
slow_launch_time 2
socket /var/lib/mysql/mysql.sock
sort_buffer_size 2097144
sql_mode
sql_notes ON
sql_warnings ON
storage_engine MyISAM
sync_binlog 0
sync_frm ON
sync_replication 0
sync_replication_slave_id 0
sync_replication_timeout 0
system_time_zone CST
table_cache 512
table_type MyISAM
thread_cache_size 8
thread_stack 196608
time_format %H:%i:%s
time_zone SYSTEM
tmp_table_size 33554432
tmpdir
transaction_alloc_block_size 8192
transaction_prealloc_size 4096
tx_isolation REPEATABLE-READ
version 4.1.22-standard-log
version_comment MySQL Community Edition - Standard (GPL)
version_compile_machine i686
version_compile_os pc-linux-gnu
wait_timeout 120
+---------------------------------+------------------------------------------+
+----------------------------+------------+
Variable_name Value
+----------------------------+------------+
Aborted_clients 1058
Aborted_connects 330
Binlog_cache_disk_use 0
Binlog_cache_use 0
Bytes_received 1631614924
Bytes_sent 1819380579
Com_admin_commands 0
Com_alter_db 0
Com_alter_table 0
Com_analyze 0
Com_backup_table 0
Com_begin 18058
Com_change_db 1371189
Com_change_master 0
Com_check 0
Com_checksum 0
Com_commit 17989
Com_create_db 0
Com_create_function 0
Com_create_index 0
Com_create_table 0
Com_dealloc_sql 0
Com_delete 318838
Com_delete_multi 0
Com_do 0
Com_drop_db 0
Com_drop_function 0
Com_drop_index 0
Com_drop_table 0
Com_drop_user 0
Com_execute_sql 0
Com_flush 0
Com_grant 0
Com_ha_close 0
Com_ha_open 0
Com_ha_read 0
Com_help 0
Com_insert 526699
Com_insert_select 0
Com_kill 1
Com_load 0
Com_load_master_data 0
Com_load_master_table 0
Com_lock_tables 0
Com_optimize 1
Com_preload_keys 0
Com_prepare_sql 0
Com_purge 0
Com_purge_before_date 0
Com_rename_table 0
Com_repair 0
Com_replace 45562
Com_replace_select 10
Com_reset 0
Com_restore_table 0
Com_revoke 0
Com_revoke_all 0
Com_rollback 0
Com_savepoint 0
Com_select 12319941
Com_set_option 331
Com_show_binlog_events 0
Com_show_binlogs 5
Com_show_charsets 83
Com_show_collations 83
Com_show_column_types 0
Com_show_create_db 6
Com_show_create_table 5
Com_show_databases 36
Com_show_errors 0
Com_show_fields 496
Com_show_grants 5
Com_show_innodb_status 0
Com_show_keys 4542
Com_show_logs 0
Com_show_master_status 0
Com_show_ndb_status 0
Com_show_new_master 0
Com_show_open_tables 0
Com_show_privileges 0
Com_show_processlist 1697
Com_show_slave_hosts 0
Com_show_slave_status 0
Com_show_status 12069
Com_show_storage_engines 0
Com_show_tables 419
Com_show_variables 211
Com_show_warnings 0
Com_slave_start 0
Com_slave_stop 0
Com_stmt_close 0
Com_stmt_execute 0
Com_stmt_prepare 0
Com_stmt_reset 0
Com_stmt_send_long_data 0
Com_truncate 0
Com_unlock_tables 0
Com_update 2513241
Com_update_multi 0
Connections 1372471
Created_tmp_disk_tables 184930
Created_tmp_files 51840
Created_tmp_tables 267683
Delayed_errors 0
Delayed_insert_threads 0
Delayed_writes 0
Flush_commands 1
Handler_commit 18058
Handler_delete 207882
Handler_discover 0
Handler_read_first 2073056
Handler_read_key 337756118
Handler_read_next 2318777882
Handler_read_prev 1016568838
Handler_read_rnd 345180111
Handler_read_rnd_next 3091295677
Handler_rollback 11030
Handler_update 2868137
Handler_write 501762512
Key_blocks_not_flushed 0
Key_blocks_unused 24807
Key_blocks_used 324093
Key_read_requests 1792786968
Key_reads 640836
Key_write_requests 979965
Key_writes 779328
Max_used_connections 421
Not_flushed_delayed_rows 0
Open_files 573
Open_streams 0
Open_tables 512
Opened_tables 57466
Qcache_free_blocks 2390
Qcache_free_memory 18294504
Qcache_hits 10133647
Qcache_inserts 12094284
Qcache_lowmem_prunes 154025
Qcache_not_cached 225369
Qcache_queries_in_cache 6977
Qcache_total_blocks 16629
Questions 29435491
Rpl_status NULL
Select_full_join 2983
Select_full_range_join 0
Select_range 1228086
Select_range_check 0
Select_scan 1113072
Slave_open_temp_tables 0
Slave_retried_transactions 0
Slave_running OFF
Slow_launch_threads 0
Slow_queries 2474
Sort_merge_passes 118597
Sort_range 1312741
Sort_rows 2550311286
Sort_scan 247873
Table_locks_immediate 23467026
Table_locks_waited 276885
Threads_cached 0
Threads_connected 42
Threads_created 57104
Threads_running 41
Uptime 395800
+----------------------------+------------+
Uptime: 395800 Threads: 42 Questions: 29435492 Slow queries: 2474 Opens: 57466 Flush tables: 1 Open tables: 512 Queries per second avg: 74.370
mysqladmin Ver 8.41 Distrib 4.1.22, for pc-linux-gnu on i686
(C) 2000 MySQL AB & MySQL Finland AB & TCX DataKonsult AB
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL license
Server version 4.1.22-standard-log
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 4 days 13 hours 56 min 40 sec
Threads: 35 Questions: 29435570 Slow queries: 2474 Opens: 57466 Flush tables: 1 Open tables: 512 Queries per second avg: 74.370
+---------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+
Id User Host db Command Time State Info
+---------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+
1331620 root localhost tw_twforums Sleep 3
1372429 tw_tribalwar localhost tw_twforums Query 0 update INSERT INTO post
(showsignature, allowsmilie, username, userid, title, pagetext, iconid, visible,
1372432 tw_tribalwar localhost tw_twforums Sleep 0
1372434 tw_tribalwar localhost tw_twforums Query 7 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372436 tw_tribalwar localhost tw_twforums Query 7 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372437 tw_tribalwar localhost tw_tribalwar Sleep 0
1372439 tw_tribalwar localhost tw_twforums Query 6 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372440 tw_tribalwar localhost tw_twforums Query 6 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372441 tw_tribalwar localhost tw_twforums Sleep 0
1372442 tw_tribalwar localhost tw_twforums Query 6 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372443 tw_tribalwar localhost tw_twforums Query 6 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372444 tw_tribalwar localhost tw_twforums Query 5 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372445 tw_tribalwar localhost tw_twforums Query 5 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372446 tw_tribalwar localhost tw_twforums Query 5 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372447 tw_tribalwar localhost tw_twforums Sleep 0
1372448 tw_tribalwar localhost tw_twforums Query 4 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372449 tw_tribalwar localhost tw_twforums Query 4 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372450 tw_tribalwar localhost tw_twforums Sleep 0
1372451 tw_tribalwar localhost tw_twforums Sleep 0
1372452 tw_tribalwar localhost tw_twforums Query 3 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372453 tw_tribalwar localhost tw_twforums Query 3 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372454 tw_tribalwar localhost tw_twforums Sleep 0
1372455 tw_tribalwar localhost tw_twforums Query 2 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372456 tw_tribalwar localhost tw_twforums Sleep 0
1372457 tw_tribalwar localhost tw_twforums Query 0 SELECT
thread.threadid, thread.title AS threadtitle, thread.forumid, pollid, open, replycount,
1372458 tw_tribalwar localhost tw_twforums Query 2 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372460 tw_tribalwar localhost tw_twforums Sleep 0
1372461 tw_tribalwar localhost tw_twforums Sleep 0
1372462 tw_tribalwar localhost tw_twforums Query 1 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372463 tw_tribalwar localhost tw_twforums Sleep 0
1372464 tw_tribalwar localhost tw_twforums Sleep 0
1372465 tw_tribalwar localhost tw_twforums Sleep 0
1372468 tw_tribalwar localhost tw_twforums Query 1 Sorting result SELECT post.postid, post.pagetext, IFNULL( user.username , post.username ) AS username, dateline
F
1372469 root localhost Query 0 show processlist
1372470 tw_tribalwar localhost tw_twforums Sleep 0
+---------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+
MySQL 4.1.22-standard-l uptime 4 13:58:44 Tue Feb 5 12:06:55 2008
__ Key __________________________________________________ _______________
Buffer used 316.60M of 384.00M %Used: 82.45
Current 359.88M %Usage: 93.72
Write hit 20.47%
Read hit 99.96%
__ Questions __________________________________________________ _________
Total 29.45M 74.4/s
DMS 15.73M 39.7/s %Total: 53.42
QC Hits 10.14M 25.6/s 34.43
Com_ 1.43M 3.6/s 4.85
COM_QUIT 1.37M 3.5/s 4.66
+Unknown 778.81k 2.0/s 2.64
Slow (10) 2.47k 0.0/s 0.01 %DMS: 0.02 Log: ON
DMS 15.73M 39.7/s 53.42
SELECT 12.33M 31.1/s 41.85 78.35
UPDATE 2.51M 6.4/s 8.54 15.98
INSERT 527.00k 1.3/s 1.79 3.35
DELETE 318.97k 0.8/s 1.08 2.03
REPLACE 45.58k 0.1/s 0.15 0.29
Com_ 1.43M 3.6/s 4.85
change_db 1.37M 3.5/s 4.66
begin 18.08k 0.0/s 0.06
commit 18.01k 0.0/s 0.06
__ SELECT and Sort __________________________________________________ ___
Scan 1.11M 2.8/s %SELECT: 9.03
Range 1.23M 3.1/s 9.97
Full join 2.99k 0.0/s 0.02
Range check 0 0/s 0.00
Full rng join 0 0/s 0.00
Sort scan 247.91k 0.6/s
Sort range 1.31M 3.3/s
Sort mrg pass 118.62k 0.3/s
__ Query Cache __________________________________________________ _______
Memory usage 12.78M of 32.00M %Used: 39.92
Block Fragmnt 19.52%
Hits 10.14M 25.6/s
Inserts 12.10M 30.6/s
Insrt:Prune 78.57:1 30.2/s
Hit:Insert 0.84:1
__ Table Locks __________________________________________________ _______
Waited 277.13k 0.7/s %Total: 1.17
Immediate 23.48M 59.3/s
__ Tables __________________________________________________ ____________
Open 512 of 512 %Cache: 100.00
Opened 57.47k 0.1/s
__ Connections __________________________________________________ _______
Max used 421 of 500 %Max: 84.20
Total 1.37M 3.5/s
__ Created Temp __________________________________________________ ______
Disk table 184.93k 0.5/s
Table 267.74k 0.7/s Size: 32.0M
File 51.85k 0.1/s
__ Threads __________________________________________________ ___________
Running 2 of 6
Cached 3 of 8 %Hit: 95.84
Created 57.14k 0.1/s
Slow 0 0/s
__ Aborted __________________________________________________ ___________
Clients 1.06k 0.0/s
Connects 330 0.0/s
__ Bytes __________________________________________________ _____________
Sent 1.95G 4.9k/s
Received 1.63G 4.1k/s
9. We have a website, but the forums are the main traffic by about 85%. Main website uses different database on same server.
10. 300-500 users. Session timeout 600
11. http://www.tribalwar.com/staff/colosus/info.php
12. Worker Module
KeepAlive On
MaxKeepAliveRequests 2000
KeepAliveTimeout 2
StartServers 10
MaxClients 1024
MinSpareThreads 50
MaxSpareThreads 300
ThreadsPerChild 25
MaxRequestsPerChild 20000
13. We have Apache logging turned off. No files over limit.
14. Linux info
root@www [/home/colosus]# uname -a
Linux www.tribalwar.com 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 26 14:30:58 EDT 2007 i686 athlon i386 GNU/Linux
root@www [/home/colosus]# ulimit -aH
core file size (blocks, -c) 1000000
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
pending signals (-i) 1024
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 14335
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
root@www [/home/colosus]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 280
stepping : 2
cpu MHz : 2411.002
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht pni syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni
bogomips : 4825.52
processor : 1
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 280
stepping : 2
cpu MHz : 2411.002
cache size : 1024 KB
physical id : 0
siblings : 2
core id : 1
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht pni syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni
bogomips : 4821.62
processor : 2
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 280
stepping : 2
cpu MHz : 2411.002
cache size : 1024 KB
physical id : 1
siblings : 2
core id : 0
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht pni syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni
bogomips : 4821.66
processor : 3
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 280
stepping : 2
cpu MHz : 2411.002
cache size : 1024 KB
physical id : 1
siblings : 2
core id : 1
cpu cores : 2
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht pni syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni
bogomips : 4821.62
There are still more tweaks we plan to do this weekend such as re-indexing the post table and thread table and we're looking at splitting the post table and possibly adding in lighttpd for static content.
I know there's a lot of talk on these forums about just throwing hardware at issues, but I do feel like there's a lot of tweaks and improvements that can be made to increase performance without costing a lot of money. The only hardware upgrade I think we need is adding a bit more RAM.
Thanks for the advice though.
I really think it has to do with the fact that our post table is 5gb, but we really don't want to prune our old threads as there's a lot of history there. Still looking for any advice you have.
Try the following in this exact order. You can ignore any of the suggestions that you have already done.
1. Upgrade MySQL server to 4.1.22
2. Upgrade PHP to 5.2.5 if you're on 5.x or PHP 4.4.8 if you're on PHP 4.x after upgrading MySQL (ensure you install as ISAPI and NOT CGI)
3. Edit mysql server's /etc/my.cnf or c:my.ini for windows and place the following mysql server settings in /etc/my.cnf and restart mysql server afterwards. Make sure to restart mysql server everytime you make changes to your my.cnf for the changes to take effect.
If mysql doesn't restart properly after my.cnf changes and you're on VPS server, make sure skip-innodb entry is removed or commented out from below my.cnf
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
safe-show-database
old_passwords
back_log = 75
skip-innodb
max_connections = 650
key_buffer = 400M
myisam_sort_buffer_size = 64M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 4000
thread_cache_size = 512
wait_timeout = 15
connect_timeout = 10
tmp_table_size = 64M
max_heap_table_size = 64M
max_allowed_packet = 64M
net_buffer_length = 16384
max_connect_errors = 10
thread_concurrency = 8
read_rnd_buffer_size = 786432
bulk_insert_buffer_size = 8M
query_cache_limit = 4M
query_cache_size = 64M
query_cache_type = 1
query_prealloc_size = 262144
query_alloc_block_size = 65536
transaction_alloc_block_size = 8192
transaction_prealloc_size = 4096
default-storage-engine = MyISAM
[mysqld_safe]
nice = -10
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 256M
sort_buffer = 256M
read_buffer = 64M
write_buffer = 64M
[mysqlhotcopy]
interactive-timeout
If you get mysql server gone away error messages, then keep increasing wait_timeout value in my.cnf by 60 second increments, then restart mysql after my.cnf changes for it to take effect. Test for a few days and see if you get less or eliminate that error message. If it still occurs, then keep repeating the 60 second increment until the message goes away. Each vB forum and server will have different optimal wait_timeout values depending on your vB forum traffic patterns and server hardware specifications.
4. Remove Eaccelerator or APC Cache if they're installed (check phpinfo.php url of yours to see) and instead install Xcache v1.2.1 final release or Xcache v1.2.2 http://xcache.lighttpd.net/wiki/Release-1.2.2 which seems to be a bit faster than APC Cache - http://www.vbulletin.com/forum/showthread.php?t=213267. Xcache site http://xcache.lighttpd.net/, documentation http://trac.lighttpd.net/trac/wiki/Docs and forums http://forum.lighttpd.net/forum/4
5. Upgrade to vB 3.0.17 http://www.vbulletin.com/forum/showthread.php?t=209720 if you're on vB 3.0.xx or upgrade to vB 3.5.8 http://www.vbulletin.com/forum/showthread.php?t=221903 if you're on vB 3.5.x. Upgrade to vB 3.6.8 PL2 http://www.vbulletin.com/forum/showthread.php?t=247739 if you're on vB 3.6.x
6. If you just upgraded to vB 3.5.x/3.6.x try to disable these 4 options:
Admin CP -> vBulletin Options -> Forums Home Page Options -> Display Logged in Users?
Admin CP -> vBulletin Options -> Forum Display Options (forumdisplay) -> Show Users Browsing Forums
Admin CP -> vBulletin Options -> Thread Display Options -> Show Users Browsing Thread
Admin CP -> vBulletin Options -> Message Searching Options -> Automatic Similar Thread search
7. Check phpinfo.php url to see if you have mod_gzip (Apache 1.3.x) or mod_deflate (Apache 2.x) loaded/installed (on phpinfo.php url page in browser you can do CTRL+F to bring up find prompt and type in mod_gzip or mod_deflate to quickly see). If you have either mod_gzip or mod_deflate loaded, then ensure vB admincp gzip compression is disabled since it's the same thing as mod_gzip or mod_deflate and double compression will just increase cpu loads. If you don't have mod_gzip or mod_deflate installed, then DO NOT set vB admincp gzip compression to a level higher than 1. Higher than 1, will only increase cpu loads unncessarily.
8. Edit httpd.conf values to following and restart apache
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 1
StartServers 20
MaxClients 256
MinSpareThreads 20
MaxSpareThreads 25
ThreadsPerChild 10
MaxRequestsPerChild 1000
What will help the most is an addition of another 4GB of RAM, so the database will (hopefully) fit into RAM completely - it will reduce database-induced I/O load to almost nil, eliminating the locking problems.
Then you can bump the key buffer size to the sum of all used indexes to let MySQL manage them most effectively, reducing IO and CPU load at the same time.
Before going for the RAM upgrade though, run "vmstat 1" for a few minutes during a peak load time and check the "wa" column, it's the percentage of time the systems waits for I/O. If you consistently see values over 20-30%, your system is indeed I/O limited and adding enough RAM for the database to fit in it will help tremendously.
Regardless of that, your MaxClients and Min/MaxSpareThreads values are insanely high, especially that you're using worker MPM. I think you'll be better with the traditional prefork module.
Anyway, go with 256 for MaxClients and 16/32 for Min/MaxSpareThreads, it will free some RAM to spare. Also KeepAlive should be off, no need to keep those Apache processes lingering there waiting for more requests.
Of course it will be much better to drop Apache completely and switch to PHP/FastCGI and lighttpd or nginx as the webserver.
Your my.cnf looks almost good, but you can shave off some memory by decreasing various buffer sizes. Also thread and table cache values are is too low.
[mysqld]
port = 3306
socket = /var/lib/mysql/mysql.sock
skip-locking
key_buffer = 512M
max_allowed_packet = 16M
table_cache = 1024
join_buffer_size = 1M
sort_buffer_size = 1M
read_buffer_size = 1M
# leave at default, or no more than 1M
# read_rnd_buffer_size = 1M
myisam_sort_buffer_size = 64M
thread_cache_size = 32
query_cache_size = 32M
# Try number of CPU's*2 for thread_concurrency
thread_concurrency = 8
max_connections = 500
wait_timeout = 120
max_heap_table_size = 64M
max_allowed_packet = 64M
vBulletin tweaks: switch to "CSS stylesheets as files" ASAP, and I see you have vbSEO but no fancy URLs, any reason you have it enabled? I guess it just sits there sucking the CPU right now.
Don't expect a sudden performance gain from these tweaks though, they are just bandaids until you take the plunge and upgrade (or better, split) the system.
Good luck.
I manage the forum very similar to yours in terms of number of messages, much more members and activity though.
#If you have any other info about this subject , Please add it free.# |
