Commit Graph

65 Commits

Author SHA1 Message Date
reshke abf4e369f5
Add makefile and CMakeFiles code to use custom openssl in build (#529)
* Add makefile and CMakeFiles code to use custom openssl in build

* APlyy fmt and rebase
2023-08-31 13:25:58 +05:00
Тимофей Шляпников 33647f5750
Better prometheus support (#414)
* fix collector init

* Added promhttp server

* Fix memory leak

* Fix memory leak

* Minimum viable server

* Port is now set via config

* Port is now set via config

* Port is now set via config

* Better code organisation & config procession

* Fix docs

* Fix old typo in label_count

* More reliable init

* Shuffle metrics by label for more appropriate collection

* Done with shuffling

* Done, not tested

* Done, now rebase

* Format

* Fix port

Co-authored-by: tim-shlyap <tim-shlyap@yandex-team.ru>
2022-03-25 21:21:59 +05:00
Yury Frolov 02f3144265
Enabled writing stats in Prometheus format. (#346)
* Early implementation

* Added description for gauges

* Added prometheus lib finding to Cmake

* Fixed FindProm comments

* Small changes

* free to prom_free

* Moved metrics to separate file

* Refactored & added new methods to prom_metrics

* Refactored cron.c to use prom_metrics.h

* Refactored CMake & odyssey.h

* Added new metrics

* Small refactoring

* Small method renaming

* Small fix in init

* Added methods for use in od_cron_stats_cb

* Small fixes

* Refactored metrics to use separate collectors

* Small fix

* Passing metrics to od_cron_stat_cb now.

* Removed unused imports

* Removed old od_log calls

* Removed TODO

* Revert "Removed old od_log calls"

This reverts commit 60000c8321.

* Uncommented od_log calls

* Added processed clients field to metrics

* Refactored metrics init

* Fixed write_stat

* Added method for worker stats

* Added method for worker stat to metrics header file

* Refactored worker stat method

* Reverted changes in odyssey.h

* Added writing metrics to od_worker

* Added new method to logger

Method provides writing big strings to log without formatting.

* Fixed prometheus log writing

* More fixes

* Fixed log writing

* Fixed log calls

* Added TODO

* Added assertion whether prom.h found

* Fixed no format logger method

Now logs follow log format

* Fixed logger

* Added log_stats_prom option to config

* Renamed od_logger_write_no_fmt to od_logger_write_plain

* Formatted

* More ifdefs

* Added memory deallocating in od_cron_stop

* Updated configuration.md

* Removed outdated TODO

* Changed label

* Formatted
2021-08-17 17:07:16 +05:00
kirill reshke af51fb655c
Update CMakeLists.txt 2021-04-29 12:57:05 +05:00
reshke dc44b323d1 return error if ldap connection establishment failed, rebind ldap
connection in case of server failture
2021-04-29 12:51:07 +05:00
reshke 74623edfb5 LDAP auth in od 2021-04-28 17:54:56 +05:00
kirill reshke 8a3831b082
Makefile build targets + quantileins memfree fix + online restart condition recheck (#261) 2021-01-14 11:54:16 +05:00
kirill reshke f305546e9c
fix scram build & tests, also make makefile a little bit more clear && better (#259)
* fix scram build & tests, also make makefile a little bit more clear and parametrized

* fix fmt & fmt target better

* remove shell test from tests run

Co-authored-by: reshke <Kirill Reshke>
2021-01-12 12:06:27 +05:00
kirill reshke 919c5c188a
debian infrastructure better + cmake fixes (#258)
* change debian build version

* some cmake lists fixes

Co-authored-by: reshke <Kirill Reshke>
2021-01-11 15:32:56 +05:00
kirill reshke 2d5af34b01
get rid of add_compile_definitions in cmake lists (#256)
Co-authored-by: reshke <Kirill Reshke>
2021-01-11 13:27:24 +05:00
reshke 0933d1ebb3 better 2020-12-30 18:49:34 +05:00
reshke b6447b3104 fix cmake minimum version 2020-12-17 17:58:25 +05:00
Daniil Zakhlystov e6781e8a4e
Rename OD_ escaping to MM_ && fix indentation for cmakelists (#235) 2020-11-23 18:57:56 +05:00
Daniil Zakhlystov e97f7e1f1c
Add compression toggle to config, cmake && Add dependencies to BuildDebian (#234)
* Add compression option to config

* Add BUILD_COMPRESSION option to cmake

* Escape identifiers

* Add preprocessing directives for OD_BUILD_COMPRESSION

* Add dependencies for BuildDebian

* apply fmt

Co-authored-by: root <root@ubuntu-build-machine.ru-central1.internal>
2020-11-23 16:55:42 +05:00
kirill reshke 68c1b70025
change default compiler standart (#233)
Co-authored-by: reshke <Kirill Reshke>
2020-11-23 16:42:44 +05:00
Daniil Zakhlystov 222da0c70a
Implement libpq compression (#230)
* Add zlib and zstd to cmake

* Add zpq_stream to machinarium

--
zpq_stream is used for streaming compression/decompression of Postgres protocol

* Add zpq_stream-related functions to machinarium

* Add compression set/free/read/write logic to machinarium

* Add startup packet compression var to kiwi

* Add compression_ack backend message support for kiwi

* Add compression setup logic for main odyssey project

* apply fmt

* fix format #2

Co-authored-by: root <root@ubuntu-build-machine.ru-central1.internal>
2020-11-23 13:22:17 +05:00
kirill reshke 129ec0be83
fix scram, use postgresql 13 in build & tests (#223)
* fix scram, use postgresql 13 in build & tests

* move scram test inside docker

Co-authored-by: reshke <Kirill Reshke>
2020-10-26 10:00:17 +05:00
reshke f030093484 use SO_REUSEPORT to graceful shutdown 2020-08-10 13:39:09 +05:00
kirill reshke 0709d7eb26
fix cmake for version (#201) 2020-08-04 09:00:14 +05:00
kirill reshke 8f61cf8db8
odyssey-show-version (#199) 2020-07-27 12:24:25 +05:00
kirill reshke 05b536c458
odyssey: fix modules API + make headers better (#168) 2020-06-15 13:26:58 +05:00
lowgear 11f8567e38
Fix memory leak at auth.c (#151)
* free pointer after use

* free only after use

* more accurate freeing

* use USE_SSL macro if openssl is found for correct pg sha256 structures definition
2020-06-11 15:02:48 +05:00
Islamov A bc89b6b097 conditionally use PAM 2020-06-09 19:11:02 +05:00
Georgy Rylov 626067abb3
tsan: fix data race in sleeplock (#145) 2020-04-15 16:40:01 +05:00
Andrey Borodin 5f7e3ea546 Upgrade dependency to PG11 and fix CI 2020-02-28 15:10:38 +05:00
Georgy Rylov e68ae929cd
ASAN (#120) 2020-02-17 12:22:04 +05:00
Andrey Borodin 155938d3a8 Use more hints to find PG libs 2019-12-24 15:31:22 +05:00
Dmitry Dolgov 4a09b5b7e9 Modifications to build on PG12 (#86)
Build seems to fail with PostgreSQL 12 version, due to incompatibility
after [1], and required presence of libpgport (to expose pg_sprintf for
scram implementation).

To improve the situation, introduce couple of changes:

* Rely on pg_config to get lib/include directories
* Add libpgport.a into linking
* Provide different versions of pg_b64_decode/encode depending on
PG_VERSION_NUM

[1]: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=cfc40d384ae51ea2886d599d2008ae57b529e6ea
2019-12-18 14:39:24 +05:00
Andrey Borodin 2262362349 Fix PostgreSQL linkage 2019-11-21 16:54:59 +05:00
Dima Starkov a8ca664a63 Implements SCRAM Authentication (#73) 2019-11-11 12:28:50 +03:00
Dmitry Simonenko c6542d7003 odyssey: post merge fixes 2019-09-27 16:47:45 +03:00
reshke debfd09d8e Add PAM auth support 2019-09-26 13:54:53 +05:00
Andrey Borodin 4198f2694d Restore odyssey_stress 2019-04-29 14:01:15 +05:00
Dmitry Simonenko 6c0a1f09f8 odyssey: major rework of io architecture
This patch intend is to improve overall io performance, reduce
cpu and system usage.

Machinarium:

IO read/write logic is highly simplified and now based on conditions
instead of previously made internal readahead buffer and msg
queues. This allows to give user more control over io operations and
avoid doing undesired readahead copying.

Above that, TLS implementation reworked to use async logic instead of
separate BIO layer.

Direct pooling functions removed.

New objects machine_cond_t and machine_iov_t objects.

Odyssey:

Introduced readahead, io and relay objects.

Relay allows to connect two io handles for direct data retransmission
without double copying.

Removed configuration options no longer needed:
packet_read_size, packet_write_queue
2019-01-23 18:43:52 +03:00
Dmitry Simonenko 9e37d1eff8 odyssey: add support for BoringSSL 2018-11-23 17:07:39 +03:00
Dmitry Simonenko da7e6ea7a5 odyssey: major io rework and refactoring
This patch moves Odyssey from stream based write caching
to message driven. All write operations are delayed
and happen on a group of messages.

This removes necessity to do IO and wait for write completion during calls
from SSL_write() context. Write timings should behave more predictable now and
number of used syscalls should be also decreased.

This patch also simplifies caching management. All clients are simply
reusing cached messages without prolonged contention.

Several configuration options are no longer used:
pipeline, cache, cache_chunk
2018-08-28 17:43:46 +03:00
Dmitry Simonenko b870c0e3a1 odyssey: rework odyssey_stress for new io api 2018-08-23 17:12:28 +03:00
Dmitry Simonenko 82e6c1a7a6 odyssey: remove shapito library 2018-08-23 16:24:07 +03:00
Dmitry Simonenko d158312c5c odyssey: do not force to have git around. fix #35 2018-07-11 16:50:05 +03:00
elsid 62b1c3969c
Set language for projects
To disable check for C++ compiler.
2018-06-10 12:44:51 +03:00
Dmitry Simonenko 7e636b9139 machinarium: support openssl >= 1.1.*. fix #17 2018-06-07 14:43:06 +03:00
Dmitry Simonenko e17d0a184c odyssey: support OPENSSL_ROOT_DIR. #4 2018-06-06 18:11:56 +03:00
Dmitry Simonenko 8518e9a959 odyssey: add initial odyssey test suite stub 2018-04-04 16:36:11 +03:00
Dmitry Simonenko f108dcba09 odyssey: project rename 2018-03-12 17:03:15 +03:00
Dmitry Simonenko 2ae8c3a4c0 odissey: introduce odissey-stress benchmark tool 2017-12-19 18:30:15 +03:00
Dmitry Simonenko 3aeb388bfa odissey: put add_subdirectory(sources) back 2017-08-21 17:50:22 +03:00
Dmitry Simonenko 97a9adfc7f odissey: minor CMakeLists.txt fixes 2017-08-21 17:32:08 +03:00
Dmitry Simonenko efe142999a odissey: add -DBUILD_DEBIAN to support Debian build 2017-08-21 17:23:50 +03:00
Dmitriy Sarafannikov ddc1d0a06b Debian package infrastructure
Package includes configuration files for odissey itself,
logrotate, init script. All of them generated by CMake
from templates placed in etc/ directory and added to
install section of Makefile which will be called by
debuild or another deb package building utilities.
2017-08-21 15:18:28 +03:00
Dmitry Simonenko ad8809acb2 odissey: fix jenkins compilation warnings 2017-08-17 17:13:31 +03:00