Commit Graph

248 Commits

Author SHA1 Message Date
Roman Khapov 40d95ad269
Backend connect timeout (#750)
* sources/console.c: remove unused variable

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/config.c: backend_connect_timeout_ms

To remove const from sources/backend.c + format fixes

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2025-01-13 15:55:31 +05:00
reshke f0659cdaf1
Minimal odyssey fd stat/debug (#633) 2025-01-13 15:45:32 +05:00
Roman Khapov 75a50ec9ec
sources/loop.c: fix infinite epoll (#744)
UINT32_MAX as default timeout should not be used, because this
can lead to infinite epoll_wait in cases where there are no timers
(like gracefully shutdown of the odyssey)

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-12-17 19:25:58 +05:00
Roman Khapov e8ef3513ab
run asan odyssey_test (#743)
* sources/machine.c: restore loop free

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/address: add destroy function

To fix memory leak in hba rule

Signed-off-by: rkhapov <r.khapov@ya.ru>

* run_test: run asan odyssey_test

To help find bugs in unit-tests

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-12-16 22:26:47 +05:00
Roman Khapov 806856e270
Msg leaks somewhere (#742)
* machinarium/channels: add get_size funcs

Can be used in some debuggin workarounds

Signed-off-by: rkhapov <r.khapov@ya.ru>

* machinarium/wait_list*

This patch adds mm_wait_list_t type
which will be used as condition variable everywhere it needed.

Ex.: wait_bus in router creates it's own wait_list, but with channels
and therefore it has memory leaks because of inconsistency of it usages.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/route.h: fix wait_bus inconsistency

route->wait_bus was implemented with channels.
This leads to memory leaks because of inconsystency of queued
clients notifications.

This patch replaces channel with wait_list, which will not lead
to memory leaks.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/config_reader.c: fix small memleak

just to silence asan

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-12-16 19:00:49 +05:00
Roman Khapov 8dc89ce423
machinarium/sources/tls.c: fix SIGSEGV (#725)
mm_tls_read can have io without on_read

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-12-10 15:05:05 +05:00
secwall 9ea042d36f
Fix some minor typos (#724) 2024-12-10 12:35:31 +05:00
Roman Khapov a7049a26d9
several fixes (#723)
* sources/hba_reader.c: fix list of keywords

It must contain {0,0,0} at the end.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* context.c: fix warn

This patch fixes warnings about writing function
pointer into data pointer

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/macro.h: fix warn

about signed vs unsiged comparison

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: remove unused func

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/counter.c: fix warn

About memcpy to volatile variable

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: fix warn

And remove useless if's

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: remove unused var

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/console.c: remove unused vars

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/frontend.c: remove unsed var

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: fix warns

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/backend.c: remove unused count

From functions od_backend_query*, count arg is not used

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/odyssey.h: add backend_sync

To fix warn about implicit declaration of od_backend_request_sync_point

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/address.c: fix implicit inet_pton

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/address.c: fix warn

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/clock.c: fix warn

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: fix uninitialized var

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/frontend.c: fix warn

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/scram.c: return instead of free

goto in this place will lead to free on unitialized ptr

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/scram.c: fix warn

Case when size + 1 >= INT_MAX is extremely unlikely

Signed-off-by: rkhapov <r.khapov@ya.ru>

* test/odyssey: use test macro instead of assert

More informative + do not produce warnings

Signed-off-by: rkhapov <r.khapov@ya.ru>

* thread.c: fix warn about implicit func

Signed-off-by: rkhapov <r.khapov@ya.ru>

* docker/bin/ody-stop: SIGTERM wait

This is more useful for tests: will help to find out
if gracefully termination in odyssey doestn't work.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* docker/Dockerfile: update libasan

This will possibly fix infinite DEADLYSIGNAL error

Signed-off-by: rkhapov <r.khapov@ya.ru>

* docker/entrypoint.sh: disable addr randomization

There is some bug with asan we currently using
This bug leads to infinite DEADLYSIGNAL message
on odyssey-asan runs sometimes

This patch adds some simple workaround about it

Signed-off-by: rkhapov <r.khapov@ya.ru>

* soureces/hba.c: fix reloading

It was broken, because it sets new rules list at the stack.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/rules.c: fix router closing

It should be closed after every usage.
This patch moves closing after every logging,
otherwise there will be use-after-free.

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-12-10 00:41:45 +05:00
Roman Khapov 8090ce86dd
sources/tls.c: eof error text improve (#716)
When errno is 0, it means that client closed the connection

Signed-off-by: rkhapov <r.khapov@ya.ru>
(cherry picked from commit e4021d359980ed8e6cb86271672a0895e7aa5ae7)

Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-11-18 16:36:55 +05:00
Roman Khapov 81c16a5df9
Some debug tools (#704)
* machinarium: add machine_get_backtrace*

This functions will fix bugs by printing backtraces

Signed-off-by: rkhapov <rkhapov@ya.ru>

* sources/console.c: clients table improvements

1) rename 'ptr' to 'id' which is more obvious
2) replace useless 'link' with 'ptr' which can be used for debugging
3) add column 'coro' with coroutine id of the client, which can be used for debugging too

Signed-off-by: rkhapov <rkhapov@ya.ru>

---------

Signed-off-by: rkhapov <rkhapov@ya.ru>
Co-authored-by: rkhapov <rkhapov@ya.ru>
2024-10-21 14:06:22 +05:00
Roman Khapov 2875a713bc
Keepalive usr default value (#701)
* machinarium: add advice tcp usr timeout fn

The value of tcp user timeout should be computed from keepalive values.
This patch adds advice function to machinarium for formulae of
keepalive + interval + probes - 0.5

Signed-off-by: rkhapov <r.khapov@ya.ru>

* sources/config_reader.c: use advice tcp usr timeout

Now, default value of keepalive_usr_timeout will lead to
adviced value of tcp usr timeout, computed with values of keepalive.

Signed-off-by: rkhapov <r.khapov@ya.ru>

* test_advice_keepalive_usr_timeout.c: fix milliseconds

According to `man 7 tcp`, delay and interval values are presented
seconds.
But tcp user timeout are presented in milliseconds.
So formula fixed.

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-10-15 15:28:41 +05:00
Roman Khapov d7a32b74cb
Fix ssl error success (#687)
* tls.c: remove unused switch

Subj

Signed-off-by: rkhapov <r.khapov@ya.ru>

* machinarium/tls.c: fix handling ssl error 0

That means EOF

Signed-off-by: rkhapov <r.khapov@ya.ru>

---------

Signed-off-by: rkhapov <r.khapov@ya.ru>
Co-authored-by: rkhapov <r.khapov@ya.ru>
2024-09-12 11:16:16 +05:00
reshke 7f88f2e432
Update machinarium-gdb.py: fix (#681) 2024-09-02 14:00:10 +05:00
Andrew Jackson 588b861cc1
Add SHOW HELP; command (#669)
* Add SHOW HELP; command

* Removed extra function

* Format

---------

Co-authored-by: CommanderKeynes <andrewjackson947@gmail.coma>
2024-08-17 13:17:02 +05:00
Roman Khapov a4bf150a62
epoll.c: various fixes (#658)
There were some problems with epoll_* functions.
This is cumulative patch of fixes.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: reshke <reshke@double.cloud>
(cherry picked from commit 06ea344082e989bf30b9deced5fe0a22e9e9ea91)

Co-authored-by: reshke <reshke@double.cloud>
2024-08-13 15:05:10 +05:00
Roman Khapov 535a27e73b
od_relay_read_pending_aware (#644)
* machinarium/io: add mm_io_read_pending

This function will inform the caller, if there is data in mm_io_t, that
are ready for non-blocking read now.

It will be used later in od_relay_read maybe.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

* sources/relay.h: od_relay_read_pending_aware

There is a bug: when the readahead is full and there are some pending bytes
(especially for tls read-ahead case), relay->src->on_read signal will be set to 0,
and the client will be softlocked (no on_read signal with available
bytes on it).

This commit adds additional pending aware read functions, that performs
mahcine_cond_signal when there are any pending bytes.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

---------

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-08-12 14:06:30 +05:00
Roman Khapov 73468ece46
context_stack.c: rm PROT_EXEC (#652)
PROT_EXEC allows to execute code from coroutine stack, which is useless
feature, and moreover, this is not secure.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-08-08 13:51:12 +05:00
reshke e8f89da9e7
Update machinarium-gdb.py: add ignore-errors (#643) 2024-07-30 11:34:30 +05:00
Roman Khapov a9339c1feb
machinarium-gdb.py: fix mmcoro cache (#640)
It is necessarry to invalidate frames cache after mmcoro command.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-07-29 09:53:09 +05:00
Roman Khapov 46de97824e
machinarium/sources/epoll.c: fix mask update (#641)
In mm_epoll_read_write, if arg enabled is 0, the mask of the fd is not
updated in correctly for masks with MM_R set.

It seems like there should be parentheses.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-07-28 17:55:42 +05:00
Roman Khapov 224f553d10
machinarium-gdb.py: support all for mmcoro (#639)
Now you can write mmcoro all bt, to run bt command on every coroutines.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-07-26 18:40:33 +05:00
Roman Khapov 142895e53b
various fixes for old gdb (#638)
* machinarium-gdb.py: fix types

It will not work on older version of gdb, so its better remove it.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

* machinarium-gdb.py: fix arch detection

selected_inferrior.architecture() doesn't work on old gdb versions.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

---------

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-07-26 17:53:27 +05:00
Roman Khapov c09cfa4553
machinarium-gdb.py: fix mmcoro for cores (#637)
mmcoro command doesnt works for core-files.
It happens because of setting pc ans sp registers, that doesnt work in
gdb for core files.

This patchs fixes that, by using gdb's stack unwiders.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
Co-authored-by: Roman Khapov <r.khapov@ya.ru>
2024-07-26 14:32:35 +05:00
Roman Khapov 63a1c7f2bb
third_party/mahcinarium/gdb: mm gdb support (#635)
* third_party/mahcinarium/gdb: mm gdb support

There are many situations in odyssey that must be debugged.
But now it is really hard to do it because of machinarium coroutines engine.

This patch series will add gdb support for machinarium's
coroutines in golang's goroutines style.

For the first step command 'info mmcoros' was added, and now it
can be used to display the list of running coroutines for each thread (mm_machine).

See gdb/README.md for more.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

* machinarium-gdb.py: move out function to iterate coros

All coroutines list of the current thread will be used in the next
command, to find specified coroutine.

This commit move out function to create coroutines list out of
MMCoroutines command.

Signed-off-by: Roman Khapov <r.khapov@ya.ru>

* machinarium-gdb.py: add mmcoro command

This command can be used to perform some gdb command
on the context of the coroutine.

It will be extremely helpful with debugging various problems in odyssey
logic.

Example:
(gdb) mmcoro 3 0 info stack
(gdb) mmcoro 3 0 frame apply 3 info locals

Signed-off-by: Roman Khapov <r.khapov@ya.ru>
2024-07-22 15:37:16 +05:00
mNTTAliev e3fa21a702
fix function name in assembly .type directive (#634) 2024-07-21 13:17:32 +03:00
Roman Khapov 9a6647fa33
Fix so nosigpipe (#629)
* machinarium/socket: fix enable usage

Seems like it was always set to 1 by mistake.

Signed-off-by: Roman Khapov <rkhapov@yandex-team.ru>

* sources/sighandler.c: fix SIGPIPE ignoring

Seems like it was added into mask but not in ignore_mask by mistake.

This patchs will not change any behaviour because now there is no
SIGPIPE section in switch, which handles signals in oddyssey machine.

Signed-off-by: Roman Khapov <rkhapov@yandex-team.ru>

* tests/oddyssey_test.c: add SIGPIPE ignoring

For now, tests doesn't have SIGPIPE ignoring.
This will lead to flaky behaviour for some tests (like tls unix socket),
because of the race for reading messages from closed socket.

This patch adds some new tests for closed socket reading and adds
SIGPIPE ignoring for odyssey_test.c

Signed-off-by: Roman Khapov <rkhapov@yandex-team.ru>

---------

Signed-off-by: Roman Khapov <rkhapov@yandex-team.ru>
2024-07-16 11:27:01 +05:00
Victor 14c491efbe
Style fixes (#625)
* all infinite loops to for(;;)

* error in word

* change kebab case in test folders in snake case

* now for test ip6 use ip6-localhost for ip4 use ip4-localhost

* malloc restyle for easier finding allocation of structures

* lint
2024-07-10 15:54:57 +05:00
Andrey Borodin de02985941
Inverse priorities of new connections (#506)
* Inverse priorities of new connections

* Apply formatting

---------

Co-authored-by: root <root@odyssey-test2.ru-central1.internal>
2023-07-12 13:22:33 +05:00
reshke 39d0c3aeb3
Channel binding for SCRAM auth method (#490)
* Channel binding for SCRAM auth method

* Apply fmt
2023-07-10 14:04:48 +03:00
reshke 1cc42b1043
configure mandatory startup parameters for backend connections (#491) 2023-07-10 12:19:50 +05:00
Ilya Shipitsin ac69f62967
third_party/machinarium/sources/tls.c: add missing checks on malloc failure (#496) 2023-05-22 13:38:37 +05:00
Jayant Anand 6e9e5edb85
Bugs autocommit (#476)
* Solved the problem for packet sequence mismatch for  packet

* Corrected the forwarding for all types of packet

* Added the formating

* Changed formating

* Added clang format

* Changed format

Co-authored-by: janand <janand@yugabyte.com>
2022-11-21 18:34:24 +04:00
reshke f0ee140956
Fix memory leaks with system server shutdown, fix prepared statemnet memeory leak (#458)
Fix memory leaks with system server shutdown. 
Fix prepared statemnet memeory leak. 
Skip volatile param caching.
2022-08-23 18:39:43 +05:00
reshke 1710d40246
Target session attr feature for storage (#456)
* Target session attr feature for storage


Storage host section is now accepts a comma-separated
PostgreSQL hosts list, in format "host2,host2"
or "[host1]:port1,host2" etc.

Storage structure now holds pointer to host, which connection was acquired.
This is required to cancel`s queries to work fine.

Target session attrs policies added:

    read-write
    read-only
    any (default)

Target session attrs uses pg_is_in_recovery function to
check if host in primary or replica. in_hot_standby option is not checked yet.
2022-08-18 17:15:13 +05:00
kirill reshke 1b2edf66fe
Fix idle in transaction session timeout pgoption (#453)
FIx a typo. Advanced PGOPTIONS parsing was added between
1.1 and 1.2 odyssey versions, including idle_in_transaction_session_timeout
parameter passing. Thats means odyssey will lookup in startup parameter
key/value pairs and parse all hardcoded pgoptions, but this require full match
between hardcoded pgoption name and startup key
2022-08-08 12:57:07 +05:00
kirill reshke 42103a04b0
Add aarch64 support for machinarium context switch function (#446) 2022-07-25 12:50:55 +05:00
vadv 01ca5b345c
fix compression build (#441)
Co-authored-by: Dmitry Vasiliev <dmitrivasilyev@ozon.ru>
2022-06-23 12:30:10 +05:00
Firm 42663bb273
[#348] Add detailed descriptions for SSL_get_verify_result() errors. (#431)
* [#348] Add detailed descriptions for SSL_get_verify_result() errors.

* [#348] Fix formating in tls.c.
2022-03-30 17:10:16 +05:00
kirill reshke 8fcbabf518
Fix unnamed prepared statement polling (#416)
* Fix unnamed prepared statement polling

* Send on read signal if ssl readahead is not exhaused

* remote null byte from snprintf

Co-authored-by: Kirill Reshke <reshke@yandex-team.ru>
2022-02-14 21:37:45 +05:00
kirill reshke 252622bf71
PGOPTION for odyssey catchup timeout (#411)
* XPGOPTION for odyssey catchup timeout

* Fix opname_size

* Fix catchup timeout parsing
2022-02-09 12:14:54 +05:00
kirill reshke f9aacb2442
Fix describe portal: do not rewrite message in this case (#404)
* Fix describe portal: do not rewrite message in this case

* Remove conf changes
2022-01-20 12:26:16 +05:00
kirill reshke fade74703a
Reserve prep stmt (#397)
* Add typedef to enum declaration

* preserve prepared statement basics

* murmur hash impl

* Hash set and hash map

* Hash set and hash map

* Allow relay to rewrite packets

* Fixes

* tmp

* Fmt

* Rebase fixes

* Min working version

* Fix heap buffer overflow

* Add show server_prep_smtms command to console db

* Store prep stmts by body hash

* Fix deadlock in console show prep stmts

* Better discard

* CLeanup code

* Support for multiple override msgs

* Remove rewrite logic

* Config validation

* Fixes

* Fmt

* Deploy rewrited msgs only on sync state

* Fix memoty issues allocaltion

* Smart discard

* Fix asan

* Handle fronent close prepered statement msg

* Fix memory leak

* Fixes

* Tests

* Fix tests

* fix tests

* remove bin

* fix clang compiler warns

* Reduse matrix to 1 case, because it doesnt change anything, as tests runs inside docker
2022-01-18 14:08:15 +05:00
kirill reshke 0287a17ba5
Config override client pg options (#399)
* Alter route cmd

* Read override options from cnf

* Tests and fixs

* fixes

* Fix memory leak
2022-01-10 10:31:04 +05:00
kirill reshke d6652fb1a5
Log pg proto msgs in debug (#400)
* Log pg proto msgs in debug

* Drop kiwi_be_read_parse_dest function
2022-01-10 10:30:14 +05:00
kirill reshke 5041bddba6
Add more pg options to pass through (#398)
* Add more pg options to pass through

* fmt
2022-01-06 18:13:34 +05:00
kirill reshke 6fa3f94b6e
Improve pgoptions parsing & add more tests (#393) 2021-12-27 13:42:14 +05:00
kirill reshke 419037c4ae
Remove usage of non-thread-save function (getopt) usage (#395)
* Remove usage of non-thread-save function (getopt) usage

* Fix fmt

* Fix include popt

* shell tests for pgoptions
2021-12-26 02:18:30 +05:00
kirill reshke d215160b3a
Fix compiler warnings (#386) 2021-12-09 16:34:04 +05:00
kirill reshke ba6513323f
Add storage watchdog with standby lag polling support (#380)
* Move auth query server querying logic to separate file

* Refactor query code to support not only "auth_query" like queries

* Move storage logic to separate file & add storage watchdog

* Force several settings on starts

* Fix simple query parsing

* Fixed bugs

* Fix config reload

* Add tests

* Move cathcup logic to separate func

* Fix tests[2]

Co-authored-by: Andrey Borodin <amborodin@acm.org>
2021-12-08 15:54:43 +05:00
Heckad f4d6f4794b Add parse pgoptions and extract search_path 2021-07-07 11:32:38 +03:00