Difference between revisions of "Gedcom Import Purge"

From TNG_Wiki
Jump to: navigation, search
(added sites)
 
Line 67: Line 67:
 
<!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-cm1 mw-customtoggle-cm2" id='mw-customcollapsible-cm1' style="text-decoration:underline;color:#0645ad;">&#91;Show Details&#93;</div>
 
<!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-cm1 mw-customtoggle-cm2" id='mw-customcollapsible-cm1' style="text-decoration:underline;color:#0645ad;">&#91;Show Details&#93;</div>
 
<div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-cm2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT -->
 
<div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-cm2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT -->
!-- BUTTON AT THE TOP OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;float:right;">&#91;Hide details&#93;</div>
+
<!-- BUTTON AT THE TOP OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;float:right;">&#91;Hide details&#93;</div>
 
Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks.
 
Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks.
  
Line 91: Line 91:
 
<!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;text-align:right;">&#91;Hide details&#93;</div>
 
<!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;text-align:right;">&#91;Hide details&#93;</div>
 
</div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE -->
 
</div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE -->
+
 
 
== Related Mods ==
 
== Related Mods ==
 
I do not know of any conflicts with other mods.
 
I do not know of any conflicts with other mods.

Latest revision as of 23:03, 4 December 2019

Gedcom Import Purge
Summary Causes the Gedcom Import Process to delete Medialinks records that have been deleted from the source database, and to retain some Places records that otherwise would be purged.
Validation
Mod Updated 2 Dec 2019
Download link v12.0.0.5.zip
TNG 12.0
Older versions can be downloaded from the Revision History.
Download stats
Author(s) Robin Richmond
Homepage this page
Mod Support My Mod Support form or TNG Community Forums
Contact Developer My Mod Support form
Latest Mod v12.0.0.5
Min TNG V 10.1
Max TNG V at least 12.2
Files modified
admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, admin_editmedia.php, js/mediafind.js, js/mediautils.js, micro_medialinks.php, ajx_updateorder.php, admin_importconfig.php, admin_updateimportconfig, English/data_help.php, English cust_text.php.


Installs: rradminbranches_ajx.php, rradminbranches_lib.php.


Installs shared files: rrshared_innermodmenu2.php, rrshared_modsettingsblocks2.php, img/rrshared_wikilogo.gif
Related Mods
Notes

Purpose of the Mod

This mod changes the Gedcom Import process to

  1. Add a flag (a new database field) to Medialinks that are created by Gedcom Imports,
  2. Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and
  3. Retain certain Places that are purged by the native code.

Without this mod, the Gedcom Import Process:

  • Leaves all Medialinks intact, including those that have been removed from the source database,
  • May purge Place records or Places data that is not replaced by the Gedcom Import.

Details

Purging Places

[Show Details]

When the "pristine" (unchanged by a mod) Gedcom Import process starts - and "all current data" is being replaced - it deletes all People, Families, Events, Notes, Citations, Sources, and Repositories in the tree being loaded, assuming it will replace them with data in the Gedcom file.

It is more careful with Places, since Places can contain data that cannot be (or is oftent not) loaded from Gedcom. Thus, when it purges Place records, it retains records that contain latitude or longitude values or descriptive notes. Still, it ignores placelevels and medialinks to Places, both of which cannot be loaded from Gedcom files.

Consequently, the Gedcom Import Purge mod modifies the Places purge at the start of the Gedcom Import process so that it retains:

  • All Places when there are multiple trees, but just one Place list,
  • Places with a longitude, latitude, note, or placelevel value, and
  • Places with a Medialink.
[Hide details]

Purging Medialinks

[Show Details]

When the Gedcom Import Process purges data from the tree being loaded, it does not delete any Medialinks, presumably because

  • Many TNG admins don't count on GEDCOM to load (all) of their media items and links; they manually add other media items and links, and
  • Gedcom does not support medialinks for places or cemeteries, and any such links in the database must have been created manually, using the TNG editors

However, as a result, Media links that have been deleted from the source database (that produced the Gedcom) will remain in TNG indefinitely, unless deleted manually

Consequently, the Gedcom Import Purge mod

  1. Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process.
  2. Modifies the Gedcom Import process to:
    1. Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates and
    2. Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - but only if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,

and

  1. Modifies the Medialinks section of the Edit Media form to display and edit the "createdbygedcom" flag. The flag is turned off when new Medialinks are created, and, from that point, can be changed, though it is not likely to be changed.
[Hide details]

Citation Medialinks

[Show Details]
[Hide details]

Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks.

Now that the TNG Gedcom Import process does create citation medialinks, Gedcom files that have citation medialinks and higher-level medialinks generate redundant medialinks. In addition, (as of TNGv12.2) not all TNG program that display medialinks handle citation medialinks cleanly.

Consequently, the Gedcom Import Purge mod

  1. Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process, and
  2. Modifies the Gedcom Import process to:
    1. Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates
    2. Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,
    3. Modify the Places purge so that it retains:
      • All Places when there are multiple trees, but just one Place list,
      • Places with a longitude, latitude, note, or placelevel value, or
      • Places with a Medialink.
    4. Ignore Citation Medialinks - if the user checks the new "Ignore Citation Medialinks" checkbox in the Gedcom Import kickoff form.

Some possible next steps:

  1. The Gedcom Gedcom Converter mod should implement an option not to create those higher-level medialinks, and/or an option to remove the citation medialinks if it creates higher-level medialinks.
  2. The Gedcom Import Purge mod should (perhaps) have an option to keep citation medialinks, and to ignore redundant higher-level medialinks.
    • (Doing so might require the converter to look ahead in the Gedcom file to determine whether such redundant links exist, so it may not be practical to implement this feature.)

Finally, note that it is common for Gedcom files to have direct links from media items to sources,people, and places, independently of citation medialinks. Thus, when I speak of suppressing or removing higher-level medialinks, I am referring to the redundant ones that duplicate a citation medialink, not to the direct medialinks.

[Hide details]

Related Mods

I do not know of any conflicts with other mods.

  • Admin Media Search - This mod changes the way that TNG displays all medialinks, and it handles citation medialinks, which are a significant concern of this mod.
  • Gedcom Import Mediatype and Gedcom Import Monitor are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are coordinated with this mod, but there are 'no dependencies among them and this mod.
  • If the optional Show Mod Names mod is installed, this mod will utilize its functionality.

Installation

Files Installed

This mod has two mod subfolders:

  1. gedcom_import_purge_v12.0.0.5 - The subfolder that you expect to exist for any mod that installs files. It contains
    • rrgedcomimportpurge_dbsetup.php This is ordinarily run once, from a link in the Mod Manager, to define the new Medialinks table field that keeps track of medialinks that are created by the Gedcom Import. The field is called createdfromgedcom.
  2. The shared folder RR-shared_mod_includes_v12.0.0.3, which contains files that support mod option management in many of my mods:

The shared folder is packaged with several mods, each of which may install some or all of its files. Once a file from the shared folder has been installed (by any mod), that file will not be overwritten or removed by the subsequent installation or un-installation of any mod (including this one), nor will its presence generate any Mod Manager errors.

If you unzip mod distribution files directly into your mods folder, then the presence of this second subfolder and the installation of its files should be invisible to you. But if you generally copy mod subfolders to your mods folder, you need to make sure to copy the folder RR-shared_mod_includes_v12.0.0.2 (well, unless you are confident that it is already in your mods folder because it was part of another mod).

[Show Installation Details]

Requirements

  • A working TNG installation.
  • An installed current version of the Mod Manager.
  • You should backup files listed in the panel on the right.

Procedure

  1. Remove and delete previous version of this mod.
  2. Backup the files updated by this mod. They are listed in the panel at the upper right.
  3. Download the .zip file, Extract its .cfg file to the mods folder.
  4. Follow the normal automated installation for Mod Manager, as shown in the example Mod Manager - Installing Config Files.

Problems?

  1. Try using the Mod Manager Remove capability
  2. Contact me through My Mod Support form.
[Hide Details]

Visualizations

In the Mod Manager, After Installation, showing the database setup hyperlink.
Gedcom import purge-modman.png

You must follow the hyperlink shown above to run the setup program that creates the database field through which TNG remembers which media links were created by a Gedcom Import.

The Database setup program

Gedcom import purge-createdb.png

The Gedcom Import Kickoff Form (Admin>>Import/Export) AFTER INSTALLATION:
Gedcom import purge-after1.jpg

New or changed elements on the Gedcom Import Kickoff form:

  1. The Inner Mod Menu, which pulls down from the "Mod Information" heading, is shown in its open state.
  2. A mod option determines whether the "Import media if present" checkbox should be checked. For the Gedcom Import Purge mod to be useful in a Gedcom Import, this checkbox should be checked.
  3. Two new checkboxes. The default status of each checkbox is determined by mod options.
    1. "Purge media links..." - The primary purpose of this mod.
    2. "Suppress the creation of citation medialinks" - It all depends on your Gedcom file and how you want to handle citation medialinks.
Message As the Gedcom Import starts:

This message is visible only if you choose the "Old style import" checkbox (or, if you have installed Gedcom Import Monitor, the "Scrolling progress listing").

Gedcom import purge-purge.png

Note that the Gedcom Import process will not purge any medialinks the first time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.

Media Links in the Admin>>Media>>Edit form

The Medialinks table in the Media Edit form now has a column with a checkbox for the "From Gedcom" flag. It is checked for Media items that were loaded through a Gedcom Import process, and defaults to unchecked (and usually remains that way) when Media items are added within TNG.
Gedcom import purge-after-medialinks.jpg

The Mod Options - in a Mod Settings Block

The Mod Settings Block for Gedcom Import Purge is at Admin>>Setup>>Import Settings. You can jump straight there from the "Change Mod Options" link in the Inner Mod Menu.
Gedcom import purge-after-modsettingsblock.jpg

Revision History

Mod Version TNG Version Date Note
12.0.0.5 12.0+ 2 Dec 2019
  1. Modified the Place purge process so that it does not purge places on sites that have multiple trees and have one shared Places list.
  2. Added an Inner Mod Menu, and moved the options to an Mod Settings Block
  3. Added a form field in the Medialinks Admin>>Media>>Edit form so that admin users can set or unset the "createdfromgedcom" (aka "From Gedcom") flag.
  4. Enhanced the message that, at the beginning of the Gedcom Import process, displays the counts of places and medialinks that were purged, and medialinks that were not purged.
  5. The Gedcom Import process generates a list of the places that were purged and recreated, and purged and not recreated.
  6. Added the "Suppress the creation of citation medialinks" checkbox on the Admin Gedcom Import kickoff form.
  7. Added code to suppress citation medialinks when the checkbox is checked.
  8. Adds a Save buttons to the left of the BodyText field in the Edit Media form, so that the user doesn't always have to scroll to the top or bottom of the form to finish.
12.0.0.4 12.0+ 15 May 2018 No functional changes; made compatible with TNGv12.
10.1.0.3p 10.1-11.1 26 Mar 2017 Removed the second line from the cust_text.php target location search string
10.1.0.3 10.1-11.1 26 Mar 2017 A technical update that, mostly, just makes Show Mod Names optional, and avoids a installation conflict introduced by TNGv11.0.1. That is, this version omits the adminlib.php patch that was part of v2 of this mod, and changes this mod's database setup program so that it works independently of that adminlib.php patch (which was fixed in TNGv11.0.1).
10.1.0.2 10.1-11.0.1 25 May 2016 Removed a unneeded <script> element that incorrectly referenced an external file, and that - under rare circumstances I still don't understand - could cause the database setup program to log you out from your TNG session. Also fixes a similarly incorrect <script> element in adminlib.php.
10.1.0.1b 10.1-11.0 25 Mar 2016 Fixed an error in a JavaScript warning. Cleaned up the code. Changed the Mod Parameters to strings rather than boolean values to be more tolerant of data entry errors.
10.1.0.1a 10.1-10.1.3 21 Feb 2016 No new end-user functionality. Now depends on Show Mod Names v2+.
10.1.0.1 10.1-10.1.3 6 Feb 2016 New mod.

Sites using this mod

If you download and install this mod, please add your site to the table below.

URL User Note Mod-Version TNG-Version User-language
Robin Richmond's Genealogy Database - admin function; not visible. Robin Richmond Mod developer 10.1.0.3 11.1.0 English
Hooley Family Links Rick Hooley Public/Private See Here See Here EN
Pete's Research TBirdUK Public/Private 12.0.0.5 12.01+ English
Woking Family Tree Project TBirdUK Public/Private 12.0.0.5 12.01+ English