Skip to content
Snippets Groups Projects

Create script to index data in FAIDARE.

Merged Célia Michotey requested to merge feat/indexing_script into master

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Célia Michotey added 25 commits

    added 25 commits

    • e2c618b5...477b7b80 - 17 commits from branch master
    • f44f2eb3 - Create script to index data in FAIDARE. Update program mapping file.
    • 55736ac6 - Correct bugs in indexation script.
    • 25422e72 - Update mappings to be coherent with generated JSON format.
    • 45afc649 - Finish indexation script and update mappings.
    • 8a2dba35 - fix: Add missing fields in trial mapping
    • ec01b301 - Correct bugs.
    • 683f91ad - Increase the agg bucket size to fetch all existing groupdId, allowing to...
    • 076a3d41 - Fixes quotes.

    Compare with previous version

  • Célia Michotey resolved all threads

    resolved all threads

  • Célia Michotey added 2 commits

    added 2 commits

    • 5e05adf3 - Update mappings to match the new sources JSON schemas.
    • 2a76a3ff - Update filter request to correctly create group 0 aliases.

    Compare with previous version

  • added 1 commit

    • 7714b695 - Update tests to match mapping modifications.

    Compare with previous version

  • Célia Michotey unmarked as a Work In Progress

    unmarked as a Work In Progress

  • @raphael.flores : To merge when you are ready

  • Documentation on how to index FAIDARE data updated following the modifications of this MR : https://forgemia.inra.fr/urgi-is/team-documentation/-/wikis/Data-Indexing#index-data-in-faidare

  • @jeremy.destin could you please test this script with some data and tell me if you think that's OK?

    Especially with the last part of the doc: https://forgemia.inra.fr/urgi-is/team-documentation/-/wikis/Data-Indexing#index-data-in-faidare

    Edited by Raphaël Flores
  • Raphaël Flores added 13 commits

    added 13 commits

    • 7714b695...d0924ad7 - 2 commits from branch master
    • be50fdc1 - Create script to index data in FAIDARE. Update program mapping file.
    • 8bca3501 - Correct bugs in indexation script.
    • 01817763 - Update mappings to be coherent with generated JSON format.
    • 041092c8 - Finish indexation script and update mappings.
    • a44a4cb8 - fix: Add missing fields in trial mapping
    • 893d0113 - Correct bugs.
    • edc06304 - Increase the agg bucket size to fetch all existing groupdId, allowing to...
    • 5dc6ef52 - Fixes quotes.
    • f6517d81 - Update mappings to match the new sources JSON schemas.
    • 7e58ffc5 - Update filter request to correctly create group 0 aliases.
    • 034cfa15 - Update tests to match mapping modifications.

    Compare with previous version

  • added 1 commit

    • 7af6b6ca - Test new ES connectivity config.

    Compare with previous version

  • added 1 commit

    • 76c530c7 - Push jobs to execute on OpenStack preferably.

    Compare with previous version

  • I've been able to index correctly on beta and staging instances @openstack with the script.

    Looks good to me!

  • Raphaël Flores added 13 commits

    added 13 commits

    • 034cfa15...c819182e - 2 commits from branch master
    • db7ffa81 - Create script to index data in FAIDARE. Update program mapping file.
    • 3fa2cc6a - Correct bugs in indexation script.
    • 5dad7baf - Update mappings to be coherent with generated JSON format.
    • 2190f7ac - Finish indexation script and update mappings.
    • dcd0a6ee - fix: Add missing fields in trial mapping
    • d225c3e5 - Correct bugs.
    • c066e35b - Increase the agg bucket size to fetch all existing groupdId, allowing to...
    • e6b2ab1a - Fixes quotes.
    • 62e1b4f4 - Update mappings to match the new sources JSON schemas.
    • 64a46400 - Update filter request to correctly create group 0 aliases.
    • 21918f97 - Update tests to match mapping modifications.

    Compare with previous version

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading