Rooba
/
odyssey
mirror of https://github.com/yandex/odyssey.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,678 Commits 47 Branches 11 Tags 35 MiB
Commit Graph

403 Commits

Author SHA1 Message Date
Andrey Borodin 8abed18d69 Minimal message validation 
Prevent improperly formatted message from breaking into serious disaster
for running process.
 2019-04-29 17:14:45 +05:00
Andrey Borodin 5fa204b082 Minor fixes 2019-04-25 14:23:23 +05:00
Dmitry Simonenko 9cd12c9d4c odyssey: minor fixes and refactoring 2019-04-13 13:26:47 +03:00
Andrey Borodin c4e2cedc35 Support of replication protocol 2019-04-12 15:13:50 +05:00
Dmitry Simonenko 859401f6dc odyssey: handle relay packet SKIP during for full-packet/deploy 2019-04-07 18:35:31 +03:00
Andrey Borodin 0b8e8309c3 Prevent condition leak 2019-03-29 15:07:59 +05:00
Andrey Borodin f552c053e8 Prevent iov leak 2019-03-29 13:52:55 +05:00
Dmitry Simonenko 73f4f9e8ca odyssey: fix warnings 2019-02-10 14:27:28 +03:00
Dmitry Simonenko f0374cb4f4 odyssey: fix server attach path on stray io event 2019-02-10 13:59:48 +03:00
Andrey Borodin b7eeeed27a Wait for read signals twice 
This is a temporary solution to ensure server connection is not reattached to
the client in case after backend is ready for query in transaction pooling
mode.
 2019-02-07 17:41:09 +03:00
Andrey Borodin 76ec3581a1
Use pg_lrand48 for portability (#48) 2019-01-30 14:30:31 +05:00
Dmitry Simonenko 89f3bc5c9e odyssey: fix readahead re-usage case and event for pending tls data 2019-01-24 18:25:46 +03:00
Dmitry Simonenko 49795088f7 odyssey: optimize relay processing 2019-01-24 15:35:07 +03:00
Dmitry Simonenko 6c0a1f09f8 odyssey: major rework of io architecture 
This patch intend is to improve overall io performance, reduce
cpu and system usage.

Machinarium:

IO read/write logic is highly simplified and now based on conditions
instead of previously made internal readahead buffer and msg
queues. This allows to give user more control over io operations and
avoid doing undesired readahead copying.

Above that, TLS implementation reworked to use async logic instead of
separate BIO layer.

Direct pooling functions removed.

New objects machine_cond_t and machine_iov_t objects.

Odyssey:

Introduced readahead, io and relay objects.

Relay allows to connect two io handles for direct data retransmission
without double copying.

Removed configuration options no longer needed:
packet_read_size, packet_write_queue
 2019-01-23 18:43:52 +03:00
Andrey Borodin b6e09fa00c Implement SHOW POOLS command 2018-12-24 17:37:40 +05:00
Dmitry Simonenko 9e0152417f odyssey: rework and optimize parameters setting logic 2018-12-13 14:18:20 +03:00
Dmitry Simonenko 9d6dea5d92 odyssey: major parameter settings optimization 2018-12-12 16:07:25 +03:00
Dmitry Simonenko c4f383ccfb odyssey: introduce pool_discard configuration parameter 2018-12-11 12:57:52 +03:00
Dmitry Simonenko 47b65e8ef2 odyssey: fix show servers callback arg 2018-12-07 15:28:32 +03:00
Dmitry Simonenko 0504a34b52 odyssey: rework pool wait logic and refactor client notification 2018-12-07 15:12:06 +03:00
Dmitry Simonenko 2e03af9c88 odyssey: major router and shared logic rework 
Goal is to improve overall performance during massive peaks
of connecting clients.

This patch makes router a shared structure, instead of
being run in separate system worker thread/coroutine.

Now router and each route object is protected by
separate mutex. Worker threads directly access router object
without calls to separate thread.
 2018-12-06 17:23:15 +03:00
Dmitry Simonenko 9115d60117 machinarium: rework time update logic 2018-11-27 16:19:16 +03:00
Dmitry Simonenko 3ac120b630 odyssey: remove left-over debug 2018-11-19 18:33:19 +03:00
Dmitry Simonenko 941449dfc1 odyssey: add clients_processed stat info per worker 2018-11-19 18:32:05 +03:00
Dmitry Simonenko c61c1e81f7 machinarium: rework msg_cache/coroutine_cache per machine thread 2018-11-19 17:52:37 +03:00
Dmitry Simonenko dc3ae5ca2e odyssey: allow to set process/workers priority 2018-11-14 16:14:13 +03:00
Dmitry Simonenko 2d4f61f923 odyssey: introduce packet_write_queue 2018-11-13 18:29:25 +03:00
Dmitry Simonenko a18f2eae68 machinarium: switch to mutexes to reduce hot spots cpu usage 2018-10-26 18:41:55 +03:00
Dmitry Simonenko 79e0fd6cac odyssey: add left over packet.h for previous commit 2018-10-23 16:56:06 +03:00
Dmitry Simonenko fbc037aa23 odyssey: implement packet retransmission in chunks; packet_read_size 2018-10-23 16:51:30 +03:00
Dmitry Simonenko 14d4a7f478 odyssey: add cache_msg_gc_size option and disable default caching 2018-09-26 18:18:17 +03:00
Dmitry Simonenko cd076462eb odyssey: implement and use shared parameters state 2018-09-25 18:40:50 +03:00
Dmitry Simonenko a790378ddb odyssey: force obsolete clients to disconnect on reload 2018-09-19 13:45:10 +03:00
Dmitry Simonenko 6f1439db3d odyssey: force expire obsolete server connections 2018-09-19 13:12:48 +03:00
Dmitry Simonenko 11dce290c3 odyssey: implement config file reload. fix #38 2018-09-18 19:22:22 +03:00
Dmitry Simonenko b2c07003f1 odyssey: implement config merge method 2018-09-18 18:54:34 +03:00
Dmitry Simonenko 3079cb2636 odyssey: remove route config version id and use obsolete flag only 2018-09-18 18:26:25 +03:00
Dmitry Simonenko 6f6df24ab1 odyssey: add route obsolete 2018-09-18 18:17:04 +03:00
Dmitry Simonenko be65c36a37 odyssey: add storage and route config comparison logic 2018-09-18 17:36:20 +03:00
Dmitry Simonenko ff648f66cd odyssey: minor fixes 2018-09-14 15:18:55 +03:00
Dmitry Simonenko 5895b8d170 odyssey: rework client parameters setup logic 2018-09-13 17:43:39 +03:00
Dmitry Simonenko 75a2d44093 odyssey: fix #41: fix include crash and allow empty include files 2018-09-11 17:18:28 +03:00
Dmitry Simonenko f69912eb91 odyssey: proper msg free on startup tls error 2018-09-10 17:15:31 +03:00
Dmitry Simonenko 13ed93f464 odyssey: do not resolve ip addresses 2018-09-06 18:29:10 +03:00
Dmitry Simonenko 6303a388ec odyssey: log timings for server connect/resolve 2018-09-06 16:53:07 +03:00
Dmitry Simonenko de824a3373 odyssey: implement msg_cache gc for oversized msgs 2018-09-05 16:04:36 +03:00
Dmitry Simonenko 62deb8bddf odyssey: make packet readers per one event only 2018-09-05 15:30:53 +03:00
Dmitry Simonenko b8d65d7225 odissey: warning fixes 2018-08-31 16:18:59 +03:00
Dmitry Simonenko 4aa764c882 odyssey: add msg cache statistics print to log 2018-08-30 17:45:13 +03:00
Dmitry Simonenko c8fd785c3e odyssey: bring reworked auth_query back 2018-08-30 17:07:16 +03:00
Dmitry Simonenko d398dd9a07 odyssey: make client_fwd_error work again 2018-08-29 16:49:10 +03:00
Dmitry Simonenko 985c1f661d odyssey: fixes for console interaction 2018-08-29 16:34:38 +03:00
Dmitry Simonenko f7937e604b odyssey: minor fixes 2018-08-29 15:40:13 +03:00
Dmitry Simonenko c2bafa08f8 odyssey: fix deploy sync counter during attach 2018-08-29 14:59:29 +03:00
Dmitry Simonenko da7e6ea7a5 odyssey: major io rework and refactoring 
This patch moves Odyssey from stream based write caching
to message driven. All write operations are delayed
and happen on a group of messages.

This removes necessity to do IO and wait for write completion during calls
from SSL_write() context. Write timings should behave more predictable now and
number of used syscalls should be also decreased.

This patch also simplifies caching management. All clients are simply
reusing cached messages without prolonged contention.

Several configuration options are no longer used:
pipeline, cache, cache_chunk
 2018-08-28 17:43:46 +03:00
Dmitry Simonenko be59c897cc odyssey: gather current stats in routes instead of server 2018-08-09 17:39:36 +03:00
Dmitry Simonenko 13cd78edfb odyssey: move server sync state out of stats 2018-08-08 16:58:39 +03:00
Dmitry Simonenko 17dea01e1d odyssey: fix config error message 2018-08-08 16:27:25 +03:00
Dmitry Simonenko 3eb948af76 odyssey: proper server->tls init on server listen 2018-08-08 15:15:21 +03:00
Dmitry Simonenko 008efe6727 odyssey: make stats_interval compatible with pgbouncer 2018-08-03 17:09:37 +03:00
Dmitry Simonenko 903f496802 odyssey: rework statistics update logic 2018-08-02 19:02:02 +03:00
Dmitry Simonenko f7a4438ec1 odyssey: move stats out of server_t 2018-08-02 16:38:36 +03:00
Dmitry Simonenko e837641719 odyssey: rework query/tx stats calculation 2018-08-01 18:10:44 +03:00
Dmitry Simonenko 6e73c241f8 odyssey: implement kill_client console command. fix #39 2018-07-19 17:33:44 +03:00
Dmitry Simonenko c495b9ea8a odyssey: add unix_socket_mode configuration param 2018-07-03 16:02:46 +03:00
Dmitry Simonenko fb023d42cb odyssey: add unix socket support. fix #31 2018-07-02 18:41:17 +03:00
Dmitry Simonenko e4f0ca6d1c odyssey: fix sys/fcntl.h include warnings. #17 2018-06-13 16:51:57 +03:00
Dmitry Simonenko 977051380e odyssey: fix incorrect uses of user_len/database_len 2018-06-13 16:39:13 +03:00
Murat Kabilov 4c53cedd27
fix show stats database string format 2018-06-12 13:40:18 +02:00
Dmitry Simonenko 0b0c7ae5b1 odyssey: minor fixes 2018-06-09 13:08:43 +03:00
Dmitry Simonenko 6f80b7c678 odyssey: minor refactoring 2018-06-08 19:36:45 +03:00
Dmitry Simonenko 15a958e0e4
Merge pull request #20 from quanzl/master 
log prepared statement
 2018-06-08 19:10:32 +03:00
Dmitry Simonenko ae0aed8a55 odyssey: do not force to have default database and user. #19 2018-06-08 18:28:07 +03:00
Quan Zongliang ddf0d140a4 suppress compiler warning 
gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)

sources/config_reader.c: In function ‘od_configreader_database’:
sources/config_reader.c:234: warning: ‘token.line’ may be used uninitialized in this function
sources/config_reader.c:234: note: ‘token.line’ was declared here
 2018-06-08 12:34:21 +08:00
Quan Zongliang 32eb039908 log prepared statement 2018-06-08 12:10:30 +08:00
Dmitry Simonenko f67e0b80cc odyssey: properly validate coroutine_stack_size 2018-06-05 13:02:58 +03:00
Dmitry Simonenko e833e62b7f odyssey: set correct fd for stdout logging. fix #10 2018-06-04 15:40:10 +03:00
Dmitry Simonenko cf3e99ecca odyssey: implement coroutine_stack_size config param. #9 2018-06-04 14:51:13 +03:00
Dmitry Simonenko 8a735a8e87 odyssey: do not exit on SIGHUP 2018-05-24 19:07:57 +03:00
Dmitry Simonenko 3cb5ccbd53 odyssey: use separate stream for reset procedure 2018-05-17 15:29:12 +03:00
Dmitry Simonenko 13d97cc74d odyssey: rename main-deploy context name 2018-05-15 17:05:54 +03:00
Dmitry Simonenko f5a4bd9a35 odyssey: support configurable list of common names to check 2018-05-10 15:49:31 +03:00
Dmitry Simonenko 120ae88b3c odyssey: log tls certificate name mismatch details 2018-04-26 16:36:54 +03:00
Dmitry Simonenko a907873715 odyssey: implement authenitcation "cert" 2018-04-26 16:02:18 +03:00
Dmitry Simonenko 5ca3fa926a odyssey: update source file headers 2018-04-04 16:19:58 +03:00
Dmitry Simonenko 7cb9befec0 odyssey: add transaction counter (total_xact_count, avg_xact_count) 2018-04-03 15:03:12 +03:00
Dmitry Simonenko 7709b2546c odyssey: console: upgrade 'show stats' and make it pgbouncer 1.8.2 compatible 2018-04-02 15:41:59 +03:00
Dmitry Simonenko 8411919747 odyssey: do not expire static routes 2018-03-21 17:42:58 +03:00
Dmitry Simonenko 38430a9d51 odyssey: remove config versioning 2018-03-21 17:36:57 +03:00
Dmitry Simonenko 7be3abed51 odyssey: minor fixes 2018-03-20 17:06:30 +03:00
Dmitry Simonenko 3e88652ca0 odyssey: disable cache_chunk by default 2018-03-16 17:06:48 +03:00
Dmitry Simonenko c3ec64974e odyssey: change verbosity for incoming cancel request log msg 2018-03-15 17:45:03 +03:00
Dmitry Simonenko 3378e60827 odyssey: remove instance_t* field out from system 2018-03-13 16:37:53 +03:00
Dmitry Simonenko fb07550d06 odyssey: remove unused field 2018-03-13 16:28:45 +03:00
Dmitry Simonenko 9fcf2a4a80 odyssey: remake pooler service as system 2018-03-13 16:26:04 +03:00
Dmitry Simonenko 20a092098e odyssey: remake system_t as global_t 2018-03-13 16:17:27 +03:00
Dmitry Simonenko 5862c1762f odyssey: misc fixes 2018-03-13 15:13:42 +03:00
Dmitry Simonenko f108dcba09 odyssey: project rename 2018-03-12 17:03:15 +03:00
Dmitry Simonenko bb826db7e8 odissey: major scheme rework, rename it to config 2018-03-06 18:23:52 +03:00
Dmitry Simonenko a096d258b7 odissey: rework configuration reader 2018-03-05 17:50:51 +03:00