Optimize synchronization process

[jeanpommier]

For now (v0.2.1), when LDAP DB has hundreds of orgs & users, it takes quite some time, and possibly resources.

• benchmark sync resource consumption
• implement incremental sync: check changes in LDAP since last sync, would reduce a lot the nb of entries to sync at each run. 1 full sync run /week would be sufficient, to keep safe from divergence, and incremental syncs in-between

[landryb]

my 2c, dunno if that can help, but for a project which needed to trigger actions upon ldap changes i used https://github.com/akkornel/syncrepl, which registers as an ldap sync client; upstream is a bit abandoned, there are sometimes some weird behaviours, but all in all it does the job.

[jeanpommier]

Nice, good tip, thanks !
Actually, the reason why it's unmaintained might be because it's now part of python-ldap module (just found out, thanks to your idea): https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap-syncrepl.html

[landryb]

duh, i should have noticed that ... thanks for the pointer ;)

[landryb]

Nice, good tip, thanks !
Actually, the reason why it's unmaintained might be because it's now part of python-ldap module (just found out, thanks to your idea): https://www.python-ldap.org/en/python-ldap-3.3.0/reference/ldap-syncrepl.html

after (finally) having been able to actually look... https://github.com/akkornel/syncrepl isnt part of python-ldap at all, but it uses and exploits the SyncreplConsumer class provided by ldap.syncrepl module from python-ldap. So it lives 'on top' of python-ldap..

Compare the sample script https://github.com/python-ldap/python-ldap/blob/master/Demo/pyasn1/syncrepl.py with https://github.com/akkornel/syncrepl/blob/master/syncrepl_client/__init__.py