python-benedict/README.md

978 lines
30 KiB
Markdown
Raw Normal View History

2020-02-11 13:59:25 +00:00
[![](https://img.shields.io/pypi/pyversions/python-benedict.svg?color=blue&logo=python&logoColor=white)](https://www.python.org/)
[![](https://img.shields.io/pypi/v/python-benedict.svg?color=blue&logo=pypi&logoColor=white)](https://pypi.org/project/python-benedict/)
2022-01-27 21:43:13 +00:00
[![](https://pepy.tech/badge/python-benedict/month)](https://pepy.tech/project/python-benedict)
2023-01-20 09:42:37 +00:00
[![](https://img.shields.io/github/stars/fabiocaccamo/python-benedict?logo=github)](https://github.com/fabiocaccamo/python-benedict/stargazers)
[![](https://img.shields.io/pypi/l/python-benedict.svg?color=blue)](https://github.com/fabiocaccamo/python-benedict/blob/main/LICENSE.txt)
2019-12-16 16:40:21 +00:00
[![](https://results.pre-commit.ci/badge/github/fabiocaccamo/python-benedict/main.svg)](https://results.pre-commit.ci/latest/github/fabiocaccamo/python-benedict/main)
[![](https://img.shields.io/github/actions/workflow/status/fabiocaccamo/python-benedict/test-package.yml?branch=main&label=build&logo=github)](https://github.com/fabiocaccamo/python-benedict)
2020-02-11 13:59:25 +00:00
[![](https://img.shields.io/codecov/c/gh/fabiocaccamo/python-benedict?logo=codecov)](https://codecov.io/gh/fabiocaccamo/python-benedict)
2021-10-12 12:27:35 +00:00
[![](https://img.shields.io/codeclimate/maintainability/fabiocaccamo/python-benedict?logo=code-climate)](https://codeclimate.com/github/fabiocaccamo/python-benedict/)
2020-02-11 13:59:25 +00:00
[![](https://img.shields.io/codacy/grade/0dbd5cc2089f4dce80a0e49e6822be3c?logo=codacy)](https://www.codacy.com/app/fabiocaccamo/python-benedict)
[![](https://img.shields.io/scrutinizer/quality/g/fabiocaccamo/python-benedict?logo=scrutinizer)](https://scrutinizer-ci.com/g/fabiocaccamo/python-benedict/?branch=main)
2022-01-27 21:43:13 +00:00
[![](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
2021-10-12 12:27:35 +00:00
2019-05-17 13:04:22 +00:00
2019-05-17 11:13:15 +00:00
# python-benedict
2023-04-06 10:52:34 +00:00
python-benedict is a dict subclass with **keylist/keypath/keyattr** support, **I/O** shortcuts (`base64`, `csv`, `ini`, `json`, `pickle`, `plist`, `query-string`, `toml`, `xls`, `xml`, `yaml`) and many **utilities**... for humans, obviously.
2019-05-17 11:13:15 +00:00
2020-01-16 17:21:10 +00:00
## Features
- 100% **backward-compatible**, you can safely wrap existing dictionaries.
- `NEW` **Keyattr** support for get/set items using **keys as attributes**.
2020-02-06 14:22:52 +00:00
- **Keylist** support using **list of keys** as key.
- **Keypath** support using **keypath-separator** *(dot syntax by default)*.
- Keypath **list-index** support *(also negative)* using the standard `[n]` suffix.
- Normalized **I/O operations** with most common formats: `base64`, `csv`, `ini`, `json`, `pickle`, `plist`, `query-string`, `toml`, `xls`, `xml`, `yaml`.
- Multiple **I/O operations** backends: `file-system` *(read/write)*, `url` *(read-only)*, `s3` *(read/write)*.
2020-01-16 17:21:10 +00:00
- Many **utility** and **parse methods** to retrieve data as needed *(check the [API](#api) section)*.
2020-02-06 14:22:52 +00:00
- Well **tested**. ;)
2020-01-16 17:21:10 +00:00
## Index
- [Installation](#installation)
- [Optional Requirements](#optional-requirements)
- [Usage](#usage)
- [Basics](#basics)
2023-03-09 13:56:09 +00:00
- [Keyattr](#keyattr) `my_dict.x.y.z`
- [Keylist](#keylist) `my_dict["x", "y", "z"]`
- [Keypath](#keypath) `my_dict["x.y.z"]`
- [Custom keypath separator](#custom-keypath-separator)
2019-10-29 15:31:49 +00:00
- [Change keypath separator](#change-keypath-separator)
- [Disable keypath functionality](#disable-keypath-functionality)
2020-02-06 14:22:52 +00:00
- [List index support](#list-index-support)
2020-01-16 16:56:45 +00:00
- [API](#api)
- [Utility methods](#utility-methods)
- [I/O methods](#io-methods)
- [Parse methods](#parse-methods)
- [Testing](#testing)
- [License](#license)
2019-05-17 11:13:15 +00:00
## Installation
If you want to install **everything**:
- Run `pip install "python-benedict[all]"`
alternatively you can install the main package:
- Run `pip install python-benedict`, then install only the [optional requirements](#optional-requirements) you need.
### Optional Requirements
Here the hierarchy of possible installation targets available when running `pip install "python-benedict[...]"` *(each target installs all its sub-targets)*:
- `[all]`
- `[io]`
- `[toml]`
- `[xls]`
- `[xml]`
- `[yaml]`
- `[s3]`
2019-05-17 11:13:15 +00:00
## Usage
### Basics
`benedict` is a `dict` subclass, so it is possible to use it as a normal dictionary *(you can just cast an existing dict)*.
2019-05-17 11:13:15 +00:00
```python
from benedict import benedict
2019-09-24 14:25:53 +00:00
# create a new empty instance
2019-07-09 14:04:36 +00:00
d = benedict()
# or cast an existing dict
2019-07-09 14:04:36 +00:00
d = benedict(existing_dict)
2020-02-06 14:22:52 +00:00
# or create from data source (filepath, url or data-string) in a supported format:
# Base64, CSV, JSON, TOML, XML, YAML, query-string
2023-03-06 23:54:22 +00:00
d = benedict("https://localhost:8000/data.json", format="json")
2019-09-24 14:25:53 +00:00
# or in a Django view
params = benedict(request.GET.items())
2023-03-06 23:54:22 +00:00
page = params.get_int("page", 1)
2020-02-06 14:22:52 +00:00
```
### Keyattr
It is possible to get/set items using **keys as attributes** (dotted notation).
```python
d = benedict(keyattr_dynamic=True) # default False
d.profile.firstname = "Fabio"
d.profile.lastname = "Caccamo"
2023-03-06 23:54:22 +00:00
print(d) # -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }
```
By default, if the `keyattr_dynamic` is not explicitly set to `True`, this functionality works for get/set only already existing items.
#### Disable keyattr functionality
You can disable the keyattr functionality passing `keyattr_enabled=False` option in the constructor.
```python
d = benedict(existing_dict, keyattr_enabled=False) # default True
```
or using the `getter/setter` property.
```python
d.keyattr_enabled = False
```
#### Dynamic keyattr functionality
You can enable the dynamic attributes access functionality passing `keyattr_dynamic=True` in the constructor.
```python
d = benedict(existing_dict, keyattr_dynamic=True) # default False
```
or using the `getter/setter` property.
```python
d.keyattr_dynamic = True
```
2023-03-09 11:08:13 +00:00
> **Warning** - even if this feature is very useful, it has some obvious limitations: it works only for string keys that are *unprotected* (not starting with an `_`) and that don't clash with the currently supported methods names.
2020-02-06 14:22:52 +00:00
### Keylist
Wherever a **key** is used, it is possible to use also a **list (or a tuple) of keys**.
```python
d = benedict()
# set values by keys list
2023-03-06 23:54:22 +00:00
d["profile", "firstname"] = "Fabio"
d["profile", "lastname"] = "Caccamo"
print(d) # -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }
print(d["profile"]) # -> { "firstname":"Fabio", "lastname":"Caccamo" }
2020-02-06 14:22:52 +00:00
# check if keypath exists in dict
2023-03-06 23:54:22 +00:00
print(["profile", "lastname"] in d) # -> True
2020-02-06 14:22:52 +00:00
# delete value by keys list
2023-03-06 23:54:22 +00:00
del d["profile", "lastname"]
print(d["profile"]) # -> { "firstname":"Fabio" }
```
2019-07-09 14:04:36 +00:00
### Keypath
`.` is the default keypath separator.
2019-07-09 14:04:36 +00:00
If you cast an existing dict and its keys contain the keypath separator a `ValueError` will be raised.
2019-10-29 15:31:49 +00:00
In this case you should use a [custom keypath separator](#custom-keypath-separator) or [disable keypath functionality](#disable-keypath-functionality).
2019-07-09 14:04:36 +00:00
```python
2019-05-17 11:13:15 +00:00
d = benedict()
2019-07-10 09:44:04 +00:00
# set values by keypath
2023-03-06 23:54:22 +00:00
d["profile.firstname"] = "Fabio"
d["profile.lastname"] = "Caccamo"
print(d) # -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }
print(d["profile"]) # -> { "firstname":"Fabio", "lastname":"Caccamo" }
2019-07-10 09:44:04 +00:00
# check if keypath exists in dict
2023-03-06 23:54:22 +00:00
print("profile.lastname" in d) # -> True
2019-07-10 09:44:04 +00:00
# delete value by keypath
2023-03-06 23:54:22 +00:00
del d["profile.lastname"]
2019-05-17 11:13:15 +00:00
```
2019-07-09 14:04:36 +00:00
#### Custom keypath separator
You can customize the keypath separator passing the `keypath_separator` argument in the constructor.
2019-10-29 15:38:51 +00:00
2019-10-29 15:31:49 +00:00
If you pass an existing dict to the constructor and its keys contain the keypath separator an `Exception` will be raised.
2019-07-09 14:04:36 +00:00
```python
2023-03-06 23:54:22 +00:00
d = benedict(existing_dict, keypath_separator="/")
2019-07-09 14:04:36 +00:00
```
2019-10-29 15:31:49 +00:00
#### Change keypath separator
You can change the `keypath_separator` at any time using the `getter/setter` property.
2019-10-29 15:38:51 +00:00
2019-10-29 15:31:49 +00:00
If any existing key contains the new `keypath_separator` an `Exception` will be raised.
```python
2023-03-06 23:54:22 +00:00
d.keypath_separator = "/"
2019-10-29 15:31:49 +00:00
```
#### Disable keypath functionality
You can disable the keypath functionality passing `keypath_separator=None` option in the constructor.
2019-07-09 14:04:36 +00:00
```python
2019-09-20 14:27:16 +00:00
d = benedict(existing_dict, keypath_separator=None)
2019-07-09 14:04:36 +00:00
```
or using the `getter/setter` property.
2019-10-29 15:31:49 +00:00
```python
d.keypath_separator = None
```
2020-02-06 14:22:52 +00:00
#### List index support
List index are supported, keypaths can include indexes *(also negative)* using `[n]`, to perform any operation very fast:
```python
# Eg. get last location cordinates of the first result:
2023-03-06 23:54:22 +00:00
loc = d["results[0].locations[-1].coordinates"]
lat = loc.get_decimal("latitude")
lng = loc.get_decimal("longitude")
2020-02-06 14:22:52 +00:00
```
2020-01-16 16:56:45 +00:00
### API
2020-01-16 17:26:34 +00:00
- **Utility methods**
2020-01-16 16:25:14 +00:00
- [`clean`](#clean)
- [`clone`](#clone)
- [`dump`](#dump)
- [`filter`](#filter)
2020-08-27 13:13:18 +00:00
- [`find`](#find)
2020-01-16 16:25:14 +00:00
- [`flatten`](#flatten)
2020-02-06 14:22:52 +00:00
- [`groupby`](#groupby)
2020-01-16 16:25:14 +00:00
- [`invert`](#invert)
- [`items_sorted_by_keys`](#items_sorted_by_keys)
- [`items_sorted_by_values`](#items_sorted_by_values)
- [`keypaths`](#keypaths)
2020-09-22 12:10:59 +00:00
- [`match`](#match)
2020-01-16 16:25:14 +00:00
- [`merge`](#merge)
- [`move`](#move)
2020-02-06 14:22:52 +00:00
- [`nest`](#nest)
2020-01-16 16:25:14 +00:00
- [`remove`](#remove)
- [`rename`](#rename)
- [`search`](#search)
- [`standardize`](#standardize)
- [`subset`](#subset)
- [`swap`](#swap)
- [`traverse`](#traverse)
- [`unflatten`](#unflatten)
- [`unique`](#unique)
2020-01-16 16:56:45 +00:00
2020-01-16 17:26:34 +00:00
- **I/O methods**
2020-01-16 16:56:45 +00:00
2020-01-16 16:25:14 +00:00
- [`from_base64`](#from_base64)
- [`from_csv`](#from_csv)
2021-05-04 21:22:11 +00:00
- [`from_ini`](#from_ini)
2020-01-16 16:25:14 +00:00
- [`from_json`](#from_json)
2020-02-21 12:55:27 +00:00
- [`from_pickle`](#from_pickle)
2020-09-09 14:45:39 +00:00
- [`from_plist`](#from_plist)
2020-01-16 16:25:14 +00:00
- [`from_query_string`](#from_query_string)
- [`from_toml`](#from_toml)
- [`from_xls`](#from_xls)
2020-01-16 16:25:14 +00:00
- [`from_xml`](#from_xml)
- [`from_yaml`](#from_yaml)
- [`to_base64`](#to_base64)
- [`to_csv`](#to_csv)
2021-05-04 21:22:11 +00:00
- [`to_ini`](#to_ini)
2020-01-16 16:25:14 +00:00
- [`to_json`](#to_json)
2020-02-21 12:55:27 +00:00
- [`to_pickle`](#to_pickle)
2020-09-09 14:45:39 +00:00
- [`to_plist`](#to_plist)
2020-01-16 16:25:14 +00:00
- [`to_query_string`](#to_query_string)
- [`to_toml`](#to_toml)
- [`to_xml`](#to_xml)
- [`to_yaml`](#to_yaml)
2020-01-16 16:56:45 +00:00
2020-01-16 17:26:34 +00:00
- **Parse methods**
2020-01-16 16:56:45 +00:00
2020-01-16 16:25:14 +00:00
- [`get_bool`](#get_bool)
- [`get_bool_list`](#get_bool_list)
- [`get_date`](#get_date)
- [`get_date_list`](#get_date_list)
2020-01-16 16:25:14 +00:00
- [`get_datetime`](#get_datetime)
- [`get_datetime_list`](#get_datetime_list)
- [`get_decimal`](#get_decimal)
- [`get_decimal_list`](#get_decimal_list)
- [`get_dict`](#get_dict)
- [`get_email`](#get_email)
- [`get_float`](#get_float)
- [`get_float_list`](#get_float_list)
- [`get_int`](#get_int)
- [`get_int_list`](#get_int_list)
- [`get_list`](#get_list)
- [`get_list_item`](#get_list_item)
- [`get_phonenumber`](#get_phonenumber)
- [`get_slug`](#get_slug)
- [`get_slug_list`](#get_slug_list)
- [`get_str`](#get_str)
- [`get_str_list`](#get_str_list)
2020-10-15 12:19:12 +00:00
- [`get_uuid`](#get_uuid)
- [`get_uuid_list`](#get_uuid_list)
2020-01-16 16:25:14 +00:00
2020-01-16 16:56:45 +00:00
### Utility methods
2019-10-03 16:49:01 +00:00
These methods are common utilities that will speed up your everyday work.
2020-01-16 16:56:45 +00:00
Utilities that accept key argument(s) also support keypath(s).
2019-10-04 13:55:32 +00:00
Utilities that return a dictionary always return a new `benedict` instance.
2019-10-04 08:45:31 +00:00
2023-04-17 09:59:29 +00:00
#### `clean`
2019-10-03 16:49:01 +00:00
```python
2023-03-06 23:54:22 +00:00
# Clean the current dict instance removing all empty values: None, "", {}, [], ().
2020-01-30 14:34:37 +00:00
# If strings or collections (dict, list, set, tuple) flags are False,
# related empty values will not be deleted.
d.clean(strings=True, collections=True)
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `clone`
2019-10-03 16:49:01 +00:00
```python
# Return a clone (deepcopy) of the dict.
c = d.clone()
```
2023-04-17 09:59:29 +00:00
#### `dump`
2019-10-03 16:49:01 +00:00
```python
# Return a readable representation of any dict/list.
# This method can be used both as static method or instance method.
s = benedict.dump(d.keypaths())
print(s)
# or
d = benedict()
print(d.dump())
```
2023-04-17 09:59:29 +00:00
#### `filter`
2019-10-03 16:49:01 +00:00
```python
# Return a filtered dict using the given predicate function.
# Predicate function receives key, value arguments and should return a bool value.
predicate = lambda k, v: v is not None
f = d.filter(predicate)
```
2023-04-17 09:59:29 +00:00
#### `find`
2020-08-27 13:13:18 +00:00
```python
# Return the first match searching for the given keys/keypaths.
# If no result found, default value is returned.
2023-03-06 23:54:22 +00:00
keys = ["a.b.c", "m.n.o", "x.y.z"]
2020-08-27 13:13:18 +00:00
f = d.find(keys, default=0)
```
2023-04-17 09:59:29 +00:00
#### `flatten`
2019-10-03 16:49:01 +00:00
```python
2020-01-13 13:56:12 +00:00
# Return a new flattened dict using the given separator to join nested dict keys to flatten keypaths.
2023-03-06 23:54:22 +00:00
f = d.flatten(separator="_")
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `groupby`
2020-02-06 14:22:52 +00:00
```python
# Group a list of dicts at key by the value of the given by_key and return a new dict.
2023-03-06 23:54:22 +00:00
g = d.groupby("cities", by_key="country_code")
2020-02-06 14:22:52 +00:00
```
2023-04-17 09:59:29 +00:00
#### `invert`
2019-10-03 16:49:01 +00:00
```python
2019-10-04 13:55:32 +00:00
# Return an inverted dict where values become keys and keys become values.
# Since multiple keys could have the same value, each value will be a list of keys.
# If flat is True each value will be a single value (use this only if values are unique).
2019-10-03 16:49:01 +00:00
i = d.invert(flat=False)
```
2023-04-17 09:59:29 +00:00
#### `items_sorted_by_keys`
2019-10-03 16:49:01 +00:00
```python
2019-10-04 08:45:31 +00:00
# Return items (key/value list) sorted by keys.
2019-10-04 13:55:32 +00:00
# If reverse is True, the list will be reversed.
2019-10-03 16:49:01 +00:00
items = d.items_sorted_by_keys(reverse=False)
```
2023-04-17 09:59:29 +00:00
#### `items_sorted_by_values`
2019-10-03 16:49:01 +00:00
```python
2019-10-04 08:45:31 +00:00
# Return items (key/value list) sorted by values.
# If reverse is True, the list will be reversed.
2019-10-03 16:49:01 +00:00
items = d.items_sorted_by_values(reverse=False)
```
2023-04-17 09:59:29 +00:00
#### `keypaths`
2019-10-29 15:31:49 +00:00
```python
# Return a list of all keypaths in the dict.
# If indexes is True, the output will include list values indexes.
k = d.keypaths(indexes=False)
2019-10-29 15:31:49 +00:00
```
2023-04-17 09:59:29 +00:00
#### `match`
2020-09-22 12:10:59 +00:00
```python
# Return a list of all values whose keypath matches the given pattern (a regex or string).
# If pattern is string, wildcard can be used (eg. [*] can be used to match all list indexes).
# If indexes is True, the pattern will be matched also against list values.
2020-09-23 09:10:08 +00:00
m = d.match(pattern, indexes=True)
2020-09-22 12:10:59 +00:00
```
2023-04-17 09:59:29 +00:00
#### `merge`
2019-10-03 16:49:01 +00:00
```python
# Merge one or more dictionary objects into current instance (deepupdate).
# Sub-dictionaries keys will be merged together.
# If overwrite is False, existing values will not be overwritten.
# If concat is True, list values will be concatenated together.
2020-12-09 10:03:03 +00:00
d.merge(a, b, c, overwrite=True, concat=False)
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `move`
2019-10-04 13:55:32 +00:00
```python
# Move an item from key_src to key_dst.
# It can be used to rename a key.
2019-10-14 12:47:49 +00:00
# If key_dst exists, its value will be overwritten.
2023-03-06 23:54:22 +00:00
d.move("a", "b", overwrite=True)
2019-10-04 13:55:32 +00:00
```
2023-04-17 09:59:29 +00:00
#### `nest`
2020-02-06 14:22:52 +00:00
```python
# Nest a list of dicts at the given key and return a new nested list
# using the specified keys to establish the correct items hierarchy.
2023-03-06 23:54:22 +00:00
d.nest("values", id_key="id", parent_id_key="parent_id", children_key="children")
2020-02-06 14:22:52 +00:00
```
2023-04-17 09:59:29 +00:00
#### `remove`
2019-10-03 16:49:01 +00:00
```python
# Remove multiple keys from the dict.
2019-10-04 13:55:32 +00:00
# It is possible to pass a single key or more keys (as list or *args).
2023-03-06 23:54:22 +00:00
d.remove(["firstname", "lastname", "email"])
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `rename`
2020-01-13 13:56:12 +00:00
```python
2023-03-06 23:54:22 +00:00
# Rename a dict item key from "key" to "key_new".
2020-01-13 13:56:12 +00:00
# If key_new exists, a KeyError will be raised.
2023-03-06 23:54:22 +00:00
d.rename("first_name", "firstname")
2020-01-13 13:56:12 +00:00
```
2023-04-17 09:59:29 +00:00
#### `search`
2020-01-13 13:56:12 +00:00
```python
# Search and return a list of items (dict, key, value, ) matching the given query.
2023-03-06 23:54:22 +00:00
r = d.search("hello", in_keys=True, in_values=True, exact=False, case_sensitive=False)
2020-01-13 13:56:12 +00:00
```
2023-04-17 09:59:29 +00:00
#### `standardize`
2019-10-29 15:31:49 +00:00
```python
# Standardize all dict keys, e.g. "Location Latitude" -> "location_latitude".
d.standardize()
```
2023-04-17 09:59:29 +00:00
#### `subset`
2019-10-03 16:49:01 +00:00
```python
# Return a dict subset for the given keys.
2019-10-04 13:55:32 +00:00
# It is possible to pass a single key or more keys (as list or *args).
2023-03-06 23:54:22 +00:00
s = d.subset(["firstname", "lastname", "email"])
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `swap`
2019-10-04 13:55:32 +00:00
```python
# Swap items values at the given keys.
2023-03-06 23:54:22 +00:00
d.swap("firstname", "lastname")
2019-10-04 13:55:32 +00:00
```
2023-04-17 09:59:29 +00:00
#### `traverse`
2019-10-29 15:31:49 +00:00
```python
# Traverse a dict passing each item (dict, key, value) to the given callback function.
def f(d, key, value):
2023-03-06 23:54:22 +00:00
print(f"dict: {d} - key: {key} - value: {value}")
2019-10-29 15:31:49 +00:00
d.traverse(f)
```
2023-04-17 09:59:29 +00:00
#### `unflatten`
2020-01-13 13:56:12 +00:00
```python
# Return a new unflattened dict using the given separator to split dict keys to nested keypaths.
2023-03-06 23:54:22 +00:00
u = d.unflatten(separator="_")
2020-01-13 13:56:12 +00:00
```
2023-04-17 09:59:29 +00:00
#### `unique`
2019-10-14 12:47:49 +00:00
```python
# Remove duplicated values from the dict.
d.unique()
```
2020-01-16 16:56:45 +00:00
### I/O methods
2019-09-24 14:25:53 +00:00
2023-03-06 23:54:22 +00:00
It is possible to create a `benedict` instance directly from data-source (`filepath`, `url`, `s3` or `data-string`) by passing the data source and the data format (optional, default "json") in the constructor.
2019-09-24 14:25:53 +00:00
```python
# filepath
2023-03-06 23:54:22 +00:00
d = benedict("/root/data.yml", format="yaml")
2019-09-24 14:25:53 +00:00
# url
2023-03-06 23:54:22 +00:00
d = benedict("https://localhost:8000/data.xml", format="xml")
2019-09-24 14:25:53 +00:00
# s3
2023-03-06 23:54:22 +00:00
d = benedict("s3://my-bucket/data.xml", s3_options={"aws_access_key_id": "...", "aws_secret_access_key": "..."})
2019-09-24 14:25:53 +00:00
# data-string
d = benedict('{"a": 1, "b": 2, "c": 3, "x": 7, "y": 8, "z": 9}')
```
These methods simplify I/O operations with most common formats: `base64`, `csv`, `ini`, `json`, `pickle`, `plist`, `query-string`, `toml`, `xls`, `xml`, `yaml`.
2019-11-07 16:52:13 +00:00
In all `from_*` methods, the first argument can be: **url**, **filepath** or **data-string**.
2023-03-06 23:54:22 +00:00
In all `to_*` methods, if `filepath="..."` kwarg is specified, the output will be also **saved** at the specified filepath.
2019-10-03 16:49:01 +00:00
2023-04-17 09:59:29 +00:00
#### `from_base64`
2019-10-03 16:49:01 +00:00
```python
# Try to load/decode a base64 encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
2020-02-21 12:55:27 +00:00
# It's possible to choose the subformat used under the hood:
2023-03-06 23:54:22 +00:00
# ('csv', 'json', 'query-string', 'toml', 'xml', 'yaml'), default: 'json'.
2019-11-07 16:52:13 +00:00
# It's possible to choose the encoding, default 'utf-8'.
2019-10-03 16:49:01 +00:00
# A ValueError is raised in case of failure.
2023-03-06 23:54:22 +00:00
d = benedict.from_base64(s, subformat="json", encoding="utf-8", **kwargs)
2019-11-07 16:52:13 +00:00
```
2023-04-17 09:59:29 +00:00
#### `from_csv`
2019-11-07 16:52:13 +00:00
```python
# Try to load/decode a csv encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
2019-11-07 16:52:13 +00:00
# It's possible to specify the columns list, default: None (in this case the first row values will be used as keys).
2020-02-21 12:55:27 +00:00
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/csv.html
2019-11-07 16:52:13 +00:00
# A ValueError is raised in case of failure.
d = benedict.from_csv(s, columns=None, columns_row=True, **kwargs)
2019-10-03 16:49:01 +00:00
```
2019-09-10 14:58:26 +00:00
2023-04-17 09:59:29 +00:00
#### `from_ini`
2021-05-04 21:22:11 +00:00
```python
# Try to load/decode a ini encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/configparser.html
# A ValueError is raised in case of failure.
d = benedict.from_ini(s, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `from_json`
2019-09-10 14:58:26 +00:00
```python
2019-09-24 14:25:53 +00:00
# Try to load/decode a json encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
2020-02-21 12:55:27 +00:00
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/json.html
2019-09-10 14:58:26 +00:00
# A ValueError is raised in case of failure.
2019-09-24 14:25:53 +00:00
d = benedict.from_json(s, **kwargs)
2019-09-10 14:58:26 +00:00
```
2023-04-17 09:59:29 +00:00
#### `from_pickle`
2020-02-21 12:55:27 +00:00
```python
# Try to load/decode a pickle encoded in Base64 format and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/pickle.html
# A ValueError is raised in case of failure.
d = benedict.from_pickle(s, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `from_plist`
2020-09-09 14:45:39 +00:00
```python
# Try to load/decode a p-list encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# https://docs.python.org/3/library/plistlib.html
# A ValueError is raised in case of failure.
d = benedict.from_plist(s, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `from_query_string`
2019-09-10 14:58:26 +00:00
```python
2019-12-12 17:30:21 +00:00
# Try to load/decode a query-string and return it as benedict instance.
2019-09-24 14:25:53 +00:00
# Accept as first argument: url, filepath or data-string.
2019-09-10 14:58:26 +00:00
# A ValueError is raised in case of failure.
2019-12-12 17:30:21 +00:00
d = benedict.from_query_string(s, **kwargs)
2019-09-10 14:58:26 +00:00
```
2023-04-17 09:59:29 +00:00
#### `from_toml`
2019-10-14 12:47:49 +00:00
```python
2019-12-12 17:30:21 +00:00
# Try to load/decode a toml encoded data and return it as benedict instance.
2019-10-14 12:47:49 +00:00
# Accept as first argument: url, filepath or data-string.
2020-02-21 12:55:27 +00:00
# It's possible to pass decoder specific options using kwargs:
# https://pypi.org/project/toml/
2019-10-14 12:47:49 +00:00
# A ValueError is raised in case of failure.
2019-12-12 17:30:21 +00:00
d = benedict.from_toml(s, **kwargs)
2019-10-14 12:47:49 +00:00
```
2023-04-17 09:59:29 +00:00
#### `from_xls`
```python
# Try to load/decode a xls file (".xls", ".xlsx", ".xlsm") from url, filepath or data-string.
# Accept as first argument: url, filepath or data-string.
# It's possible to pass decoder specific options using kwargs:
# - https://openpyxl.readthedocs.io/ (for .xlsx and .xlsm files)
# - https://pypi.org/project/xlrd/ (for .xls files)
# A ValueError is raised in case of failure.
d = benedict.from_xls(s, sheet=0, columns=None, columns_row=True, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `from_xml`
```python
2019-09-24 14:25:53 +00:00
# Try to load/decode a xml encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
2020-02-21 12:55:27 +00:00
# It's possible to pass decoder specific options using kwargs:
# https://github.com/martinblech/xmltodict
# A ValueError is raised in case of failure.
2019-09-24 14:25:53 +00:00
d = benedict.from_xml(s, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `from_yaml`
```python
2019-09-24 14:25:53 +00:00
# Try to load/decode a yaml encoded data and return it as benedict instance.
# Accept as first argument: url, filepath or data-string.
2020-02-21 12:55:27 +00:00
# It's possible to pass decoder specific options using kwargs:
# https://pyyaml.org/wiki/PyYAMLDocumentation
# A ValueError is raised in case of failure.
2019-09-24 14:25:53 +00:00
d = benedict.from_yaml(s, **kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_base64`
2019-10-03 16:49:01 +00:00
```python
2019-11-07 16:52:13 +00:00
# Return the dict instance encoded in base64 format and optionally save it at the specified 'filepath'.
2020-02-21 12:55:27 +00:00
# It's possible to choose the subformat used under the hood:
2023-03-06 23:54:22 +00:00
# ('csv', json', 'query-string', 'toml', 'xml', 'yaml'), default: 'json'.
2019-11-07 16:52:13 +00:00
# It's possible to choose the encoding, default 'utf-8'.
2019-10-14 12:47:49 +00:00
# It's possible to pass decoder specific options using kwargs.
2019-10-03 16:49:01 +00:00
# A ValueError is raised in case of failure.
2023-03-06 23:54:22 +00:00
s = d.to_base64(subformat="json", encoding="utf-8", **kwargs)
2019-11-07 16:52:13 +00:00
```
2023-04-17 09:59:29 +00:00
#### `to_csv`
2019-11-07 16:52:13 +00:00
```python
# Return a list of dicts in the current dict encoded in csv format and optionally save it at the specified filepath.
2019-11-07 16:52:13 +00:00
# It's possible to specify the key of the item (list of dicts) to encode, default: 'values'.
# It's possible to specify the columns list, default: None (in this case the keys of the first item will be used).
# A ValueError is raised in case of failure.
2023-03-06 23:54:22 +00:00
s = d.to_csv(key="values", columns=None, columns_row=True, **kwargs)
2019-10-03 16:49:01 +00:00
```
2023-04-17 09:59:29 +00:00
#### `to_ini`
2021-05-04 21:22:11 +00:00
```python
# Return the dict instance encoded in ini format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/configparser.html
# A ValueError is raised in case of failure.
s = d.to_ini(**kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_json`
```python
# Return the dict instance encoded in json format and optionally save it at the specified filepath.
2020-02-21 12:55:27 +00:00
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/json.html
# A ValueError is raised in case of failure.
2019-11-07 16:52:13 +00:00
s = d.to_json(**kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_pickle`
2020-02-21 12:55:27 +00:00
```python
# Return the dict instance as pickle encoded in Base64 format and optionally save it at the specified filepath.
2020-02-21 13:36:07 +00:00
# The pickle protocol used by default is 2.
2020-02-21 12:55:27 +00:00
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/pickle.html
# A ValueError is raised in case of failure.
s = d.to_pickle(**kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_plist`
2020-09-09 14:45:39 +00:00
```python
# Return the dict instance encoded in p-list format and optionally save it at the specified filepath.
# It's possible to pass encoder specific options using kwargs:
# https://docs.python.org/3/library/plistlib.html
# A ValueError is raised in case of failure.
s = d.to_plist(**kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_query_string`
2019-10-14 12:47:49 +00:00
```python
# Return the dict instance as query-string and optionally save it at the specified filepath.
# A ValueError is raised in case of failure.
2019-11-07 16:52:13 +00:00
s = d.to_query_string(**kwargs)
2019-10-14 12:47:49 +00:00
```
2023-04-17 09:59:29 +00:00
#### `to_toml`
2019-07-19 09:02:18 +00:00
```python
# Return the dict instance encoded in toml format and optionally save it at the specified filepath.
2020-02-21 12:55:27 +00:00
# It's possible to pass encoder specific options using kwargs:
# https://pypi.org/project/toml/
# A ValueError is raised in case of failure.
2019-11-07 16:52:13 +00:00
s = d.to_toml(**kwargs)
2019-07-19 09:02:18 +00:00
```
2023-04-17 09:59:29 +00:00
#### `to_xml`
```python
# Return the dict instance encoded in xml format and optionally save it at the specified filepath.
2020-02-21 12:55:27 +00:00
# It's possible to pass encoder specific options using kwargs:
# https://github.com/martinblech/xmltodict
# A ValueError is raised in case of failure.
2019-11-07 16:52:13 +00:00
s = d.to_xml(**kwargs)
```
2023-04-17 09:59:29 +00:00
#### `to_yaml`
2019-07-19 09:02:18 +00:00
```python
2019-11-07 16:52:13 +00:00
# Return the dict instance encoded in yaml format.
# If filepath option is passed the output will be saved ath
2020-02-21 12:55:27 +00:00
# It's possible to pass encoder specific options using kwargs:
# https://pyyaml.org/wiki/PyYAMLDocumentation
# A ValueError is raised in case of failure.
2019-11-07 16:52:13 +00:00
s = d.to_yaml(**kwargs)
2019-07-19 09:02:18 +00:00
```
2020-01-16 16:56:45 +00:00
### Parse methods
2019-07-10 12:36:51 +00:00
These methods are wrappers of the `get` method, they parse data trying to return it in the expected type.
2019-07-09 14:04:36 +00:00
2023-04-17 09:59:29 +00:00
#### `get_bool`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as bool.
2019-05-20 15:30:01 +00:00
# Values like `1`, `true`, `yes`, `on`, `ok` will be returned as `True`.
d.get_bool(key, default=False)
```
2023-04-17 09:59:29 +00:00
#### `get_bool_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of bool values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_bool_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_date`
```python
# Get value by key or keypath trying to return it as date.
# If format is not specified it will be autodetected.
# If choices and value is in choices return value otherwise default.
d.get_date(key, default=None, format=None, choices=[])
```
2023-04-17 09:59:29 +00:00
#### `get_date_list`
```python
# Get value by key or keypath trying to return it as list of date values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_date_list(key, default=[], format=None, separator=",")
```
2023-04-17 09:59:29 +00:00
#### `get_datetime`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as datetime.
# If format is not specified it will be autodetected.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
d.get_datetime(key, default=None, format=None, choices=[])
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_datetime_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of datetime values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_datetime_list(key, default=[], format=None, separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_decimal`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as Decimal.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
2023-03-06 23:54:22 +00:00
d.get_decimal(key, default=Decimal("0.0"), choices=[])
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_decimal_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of Decimal values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_decimal_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_dict`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as dict.
# If value is a json string it will be automatically decoded.
2019-05-17 11:13:15 +00:00
d.get_dict(key, default={})
```
2023-04-17 09:59:29 +00:00
#### `get_email`
2019-05-17 11:13:15 +00:00
```python
# Get email by key or keypath and return it.
# If value is blacklisted it will be automatically ignored.
# If check_blacklist is False, it will be not ignored even if blacklisted.
2023-03-06 23:54:22 +00:00
d.get_email(key, default="", choices=None, check_blacklist=True)
```
2023-04-17 09:59:29 +00:00
#### `get_float`
```python
# Get value by key or keypath trying to return it as float.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
d.get_float(key, default=0.0, choices=[])
```
2023-04-17 09:59:29 +00:00
#### `get_float_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of float values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_float_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_int`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as int.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
d.get_int(key, default=0, choices=[])
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_int_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of int values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_int_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_list_item`
2019-06-10 16:05:28 +00:00
```python
# Get list by key or keypath and return value at the specified index.
# If separator is specified and list value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_list_item(key, index=0, default=None, separator=",")
2019-06-10 16:05:28 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_phonenumber`
```python
# Get phone number by key or keypath and return a dict with different formats (e164, international, national).
# If country code is specified (alpha 2 code), it will be used to parse phone number correctly.
d.get_phonenumber(key, country_code=None, default=None)
```
2023-04-17 09:59:29 +00:00
#### `get_slug`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as slug.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
2023-03-06 23:54:22 +00:00
d.get_slug(key, default="", choices=[])
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_slug_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of slug values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_slug_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_str`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as string.
# Encoding issues will be automatically fixed.
2019-12-12 17:30:21 +00:00
# If choices and value is in choices return value otherwise default.
2023-03-06 23:54:22 +00:00
d.get_str(key, default="", choices=[])
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_str_list`
2019-05-17 11:13:15 +00:00
```python
2019-05-17 13:04:22 +00:00
# Get value by key or keypath trying to return it as list of str values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_str_list(key, default=[], separator=",")
2019-05-17 11:13:15 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_uuid`
2020-10-15 12:19:12 +00:00
```python
# Get value by key or keypath trying to return it as valid uuid.
# If choices and value is in choices return value otherwise default.
2023-03-06 23:54:22 +00:00
d.get_uuid(key, default="", choices=[])
2020-10-15 12:19:12 +00:00
```
2023-04-17 09:59:29 +00:00
#### `get_uuid_list`
2020-10-15 12:19:12 +00:00
```python
# Get value by key or keypath trying to return it as list of valid uuid values.
# If separator is specified and value is a string it will be splitted.
2023-03-06 23:54:22 +00:00
d.get_uuid_list(key, default=[], separator=",")
2020-10-15 12:19:12 +00:00
```
## Testing
2019-12-12 17:30:21 +00:00
```bash
2022-06-15 22:40:54 +00:00
# clone repository
git clone https://github.com/fabiocaccamo/python-benedict.git && cd python-benedict
2019-12-12 17:30:21 +00:00
2022-06-15 22:40:54 +00:00
# create virtualenv and activate it
python -m venv venv && . venv/bin/activate
2019-12-12 17:30:21 +00:00
2022-06-15 22:40:54 +00:00
# upgrade pip
python -m pip install --upgrade pip
2019-12-12 17:30:21 +00:00
# install requirements
2022-06-15 22:40:54 +00:00
pip install -r requirements.txt -r requirements-test.txt
2019-12-12 17:30:21 +00:00
2023-03-13 14:26:03 +00:00
# install pre-commit to run formatters and linters
pre-commit install --install-hooks
2019-12-12 17:30:21 +00:00
# run tests using tox
tox
# or run tests using unittest
python -m unittest
```
2019-05-17 11:13:15 +00:00
## License
2019-12-13 16:55:53 +00:00
Released under [MIT License](LICENSE.txt).
2020-11-27 12:12:31 +00:00
---
2022-06-15 22:40:54 +00:00
## Supporting
- :star: Star this project on [GitHub](https://github.com/fabiocaccamo/python-benedict)
- :octocat: Follow me on [GitHub](https://github.com/fabiocaccamo)
- :blue_heart: Follow me on [Twitter](https://twitter.com/fabiocaccamo)
- :moneybag: Sponsor me on [Github](https://github.com/sponsors/fabiocaccamo)
2020-11-27 12:12:31 +00:00
## See also
2021-12-06 17:50:56 +00:00
- [`python-fontbro`](https://github.com/fabiocaccamo/python-fontbro) - friendly font operations. 🧢
2020-11-27 12:12:31 +00:00
- [`python-fsutil`](https://github.com/fabiocaccamo/python-fsutil) - file-system utilities for lazy devs. 🧟‍♂️