Default per worker 16 value should be increased to achive
more CPU utilization and client connections rate.
After testing some client connection parallelism i learn that
there should be ~ 60-80 client connections per worker to get 100% of cpu loading
(with non-localhost connections)
So this patch increases 16 to 64
benchmarks/client_max_routing might be used for future learning
Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
* Reject connection initiating in replication lag too high.
In replication lag polling feature, check replication lag on connection
startuy (before auth), and reject connection if replication lag is too high.
* Apply fmt
* Target session attr feature for storage
Storage host section is now accepts a comma-separated
PostgreSQL hosts list, in format "host2,host2"
or "[host1]:port1,host2" etc.
Storage structure now holds pointer to host, which connection was acquired.
This is required to cancel`s queries to work fine.
Target session attrs policies added:
read-write
read-only
any (default)
Target session attrs uses pg_is_in_recovery function to
check if host in primary or replica. in_hot_standby option is not checked yet.
* fix collector init
* Added promhttp server
* Fix memory leak
* Fix memory leak
* Minimum viable server
* Port is now set via config
* Port is now set via config
* Port is now set via config
* Better code organisation & config procession
* Fix docs
* Fix old typo in label_count
* More reliable init
* Shuffle metrics by label for more appropriate collection
* Done with shuffling
* Done, not tested
* Done, now rebase
* Format
* Fix port
Co-authored-by: tim-shlyap <tim-shlyap@yandex-team.ru>
* Ldapless
* Ldapless
* Console are now considering roles
* For pull from origin
* Done, but not tested
* Works, but do not drop connections to console
* Update to origin
* Added drop connections with lowered role via reload
* Format and rename
* Fix makefile & dev.conf
* Fix makefile & dev.conf
* Fix lagpolling tests
* Fix tests
* Fix tests
* Add tests & fix .conf
* Add tests & fix .conf
* Add tests & fix .conf
Co-authored-by: tim-shlyap <tim-shlyap@yandex-team.ru>
* Early implementation
* Added description for gauges
* Added prometheus lib finding to Cmake
* Fixed FindProm comments
* Small changes
* free to prom_free
* Moved metrics to separate file
* Refactored & added new methods to prom_metrics
* Refactored cron.c to use prom_metrics.h
* Refactored CMake & odyssey.h
* Added new metrics
* Small refactoring
* Small method renaming
* Small fix in init
* Added methods for use in od_cron_stats_cb
* Small fixes
* Refactored metrics to use separate collectors
* Small fix
* Passing metrics to od_cron_stat_cb now.
* Removed unused imports
* Removed old od_log calls
* Removed TODO
* Revert "Removed old od_log calls"
This reverts commit 60000c8321.
* Uncommented od_log calls
* Added processed clients field to metrics
* Refactored metrics init
* Fixed write_stat
* Added method for worker stats
* Added method for worker stat to metrics header file
* Refactored worker stat method
* Reverted changes in odyssey.h
* Added writing metrics to od_worker
* Added new method to logger
Method provides writing big strings to log without formatting.
* Fixed prometheus log writing
* More fixes
* Fixed log writing
* Fixed log calls
* Added TODO
* Added assertion whether prom.h found
* Fixed no format logger method
Now logs follow log format
* Fixed logger
* Added log_stats_prom option to config
* Renamed od_logger_write_no_fmt to od_logger_write_plain
* Formatted
* More ifdefs
* Added memory deallocating in od_cron_stop
* Updated configuration.md
* Removed outdated TODO
* Changed label
* Formatted
This patch intend is to improve overall io performance, reduce
cpu and system usage.
Machinarium:
IO read/write logic is highly simplified and now based on conditions
instead of previously made internal readahead buffer and msg
queues. This allows to give user more control over io operations and
avoid doing undesired readahead copying.
Above that, TLS implementation reworked to use async logic instead of
separate BIO layer.
Direct pooling functions removed.
New objects machine_cond_t and machine_iov_t objects.
Odyssey:
Introduced readahead, io and relay objects.
Relay allows to connect two io handles for direct data retransmission
without double copying.
Removed configuration options no longer needed:
packet_read_size, packet_write_queue
This patch moves Odyssey from stream based write caching
to message driven. All write operations are delayed
and happen on a group of messages.
This removes necessity to do IO and wait for write completion during calls
from SSL_write() context. Write timings should behave more predictable now and
number of used syscalls should be also decreased.
This patch also simplifies caching management. All clients are simply
reusing cached messages without prolonged contention.
Several configuration options are no longer used:
pipeline, cache, cache_chunk
Roman Khapov