Create script to index data in FAIDARE.

added 1 commit

• c40dfec2 - Correct bugs in indexation script.

Compare with previous version

added 25 commits

• c40dfec2...4b0b74d6 - 22 commits from branch master
• 816ddeec - Create script to index data in FAIDARE. Update program mapping file.
• 5c88c2bd - Correct bugs in indexation script.
• dd5dc504 - Update mappings to be coherent with generated JSON format.

Compare with previous version

added 1 commit

• fd96afd9 - Finish indexation script and update mappings.

Compare with previous version

added 1 commit

• e8222779 - fix: Add missing fields in trial mapping

Compare with previous version

added 1 commit

• 2b58db62 - Correct bugs.

Compare with previous version

added 2 commits

• 1baa3d96 - Increase the agg bucket size to fetch all existing groupdId, allowing to...
• e2c618b5 - Fixes quotes.

Compare with previous version

added 25 commits

• e2c618b5...477b7b80 - 17 commits from branch master
• f44f2eb3 - Create script to index data in FAIDARE. Update program mapping file.
• 55736ac6 - Correct bugs in indexation script.
• 25422e72 - Update mappings to be coherent with generated JSON format.
• 45afc649 - Finish indexation script and update mappings.
• 8a2dba35 - fix: Add missing fields in trial mapping
• ec01b301 - Correct bugs.
• 683f91ad - Increase the agg bucket size to fetch all existing groupdId, allowing to...
• 076a3d41 - Fixes quotes.

Compare with previous version

resolved all threads

added 2 commits

• 5e05adf3 - Update mappings to match the new sources JSON schemas.
• 2a76a3ff - Update filter request to correctly create group 0 aliases.

Compare with previous version

added 1 commit

• 7714b695 - Update tests to match mapping modifications.

Compare with previous version

unmarked as a Work In Progress

@raphael.flores : To merge when you are ready

Documentation on how to index FAIDARE data updated following the modifications of this MR : https://forgemia.inra.fr/urgi-is/team-documentation/-/wikis/Data-Indexing#index-data-in-faidare

@jeremy.destin could you please test this script with some data and tell me if you think that's OK?

Especially with the last part of the doc: https://forgemia.inra.fr/urgi-is/team-documentation/-/wikis/Data-Indexing#index-data-in-faidare

added 13 commits

• 7714b695...d0924ad7 - 2 commits from branch master
• be50fdc1 - Create script to index data in FAIDARE. Update program mapping file.
• 8bca3501 - Correct bugs in indexation script.
• 01817763 - Update mappings to be coherent with generated JSON format.
• 041092c8 - Finish indexation script and update mappings.
• a44a4cb8 - fix: Add missing fields in trial mapping
• 893d0113 - Correct bugs.
• edc06304 - Increase the agg bucket size to fetch all existing groupdId, allowing to...
• 5dc6ef52 - Fixes quotes.
• f6517d81 - Update mappings to match the new sources JSON schemas.
• 7e58ffc5 - Update filter request to correctly create group 0 aliases.
• 034cfa15 - Update tests to match mapping modifications.

Compare with previous version

added 1 commit

• 7af6b6ca - Test new ES connectivity config.

Compare with previous version

added 1 commit

• 76c530c7 - Push jobs to execute on OpenStack preferably.

Compare with previous version

I've been able to index correctly on beta and staging instances @openstack with the script.

Looks good to me!

added 13 commits

• 034cfa15...c819182e - 2 commits from branch master
• db7ffa81 - Create script to index data in FAIDARE. Update program mapping file.
• 3fa2cc6a - Correct bugs in indexation script.
• 5dad7baf - Update mappings to be coherent with generated JSON format.
• 2190f7ac - Finish indexation script and update mappings.
• dcd0a6ee - fix: Add missing fields in trial mapping
• d225c3e5 - Correct bugs.
• c066e35b - Increase the agg bucket size to fetch all existing groupdId, allowing to...
• e6b2ab1a - Fixes quotes.
• 62e1b4f4 - Update mappings to match the new sources JSON schemas.
• 64a46400 - Update filter request to correctly create group 0 aliases.
• 21918f97 - Update tests to match mapping modifications.

Compare with previous version