Difference between revisions of "Gedcom Import Purge"

From TNG_Wiki
Jump to navigation Jump to search
(15 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{TNGmod
 
{{TNGmod
 
| mod_name        =  Gedcom Import Purge
 
| mod_name        =  Gedcom Import Purge
|  mod_summary    = Causes the Gedcom Import Process to delete Medialinks records that have been deleted in the source database, and to retain some Places records that otherwise would be purged.
+
|  mod_summary    = Causes the Gedcom Import Process to delete Medialinks records that have been deleted from the source database, and to retain some Places records that otherwise would be purged.
 
| mod_validation  =  
 
| mod_validation  =  
| download_link  =  [[Media:gedcom_import_purge_v10.1.0.3.zip|gedcom_import_purge_v10.1.0.3.zip]] {{Tv110}}{{Tv1010}}
+
| mod_last_update = 2 Dec 2019
 +
| download_link  =  [[Media:gedcom_import_purge_v12.0.0.5.zip|v12.0.0.5.zip]]{{Tv120}}
 +
<div style='font-size:90%'>Older versions can be downloaded from the [[#Revision_History|Revision History]].</div>
 
| download_stats  =
 
| download_stats  =
 
| mod_author      = [[User:Robinrichm|Robin Richmond]]
 
| mod_author      = [[User:Robinrichm|Robin Richmond]]
| mod_url        = [http://www.robinrichmond.com/family/ Robin Richmond's Genealogy Database]
+
| mod_url        = ''this page''
 
| mod_contact    = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form]
 
| mod_contact    = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form]
 
| mod_support    = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] or [http://tng.community/ TNG Community Forums]
 
| mod_support    = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] or [http://tng.community/ TNG Community Forums]
| mod_version    = 10.1.0.3
+
| mod_version    = v12.0.0.5
 
| min_TNG_ver    = 10.1
 
| min_TNG_ver    = 10.1
| max_TNG_ver    = <span style='background-color: #FFBB33; color: #000000'>11.1</span>
+
| max_TNG_ver    = at least 12.2
|  TNG_file_list  = admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, English/data_help.php, English cust_text.php
+
|  TNG_file_list  = admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, admin_editmedia.php, js/mediafind.js, js/mediautils.js, micro_medialinks.php, ajx_updateorder.php,  admin_importconfig.php, admin_updateimportconfig, English/data_help.php, English cust_text.php.
| related_mods  = [[Blue Info Button]], [[Show Mod Names]], [[Gedcom Import Mediatype]], [[Gedcom Import Monitor]] (See "Related Mods" heading)
+
<br>Installs: rradminbranches_ajx.php, rradminbranches_lib.php.
 +
<br>Installs shared files: rrshared_innermodmenu2.php, rrshared_modsettingsblocks2.php, img/rrshared_wikilogo.gif
 +
| related_mods  = [[Admin Media Search]], [[Gedcom Import Mediatype]], [[Gedcom Import Monitor]], [[Show Mod Names]]
 
| notes    =   
 
| notes    =   
}}
+
}}== Purpose of the Mod ==
 
+
This mod changes the Gedcom Import process to
 +
# Add a flag (a new database field) to Medialinks that are created by Gedcom Imports,
 +
# Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and
 +
# Retain certain Places that are purged by the native code.
 
{| style="margin-right:0.5 em;" align="right"  
 
{| style="margin-right:0.5 em;" align="right"  
 
| __TOC__
 
| __TOC__
 +
|}
 +
Without this mod, the Gedcom Import Process:
 +
* Leaves ''all'' Medialinks intact, including those that have been removed from the source database,
 +
* May purge Place records or Places data that is not replaced by the Gedcom Import.
  
|}
+
== Details ==
 +
=== Purging Places ===
 +
<!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-places1 mw-customtoggle-places2" id='mw-customcollapsible-places1' style="text-decoration:underline;color:#0645ad;">&#91;Show Details&#93;</div>
 +
<div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-places2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT -->
 +
When the "pristine" (unchanged by a mod) Gedcom Import process starts - and "all current data" is being replaced - it deletes all People, Families, Events, Notes, Citations, Sources, and Repositories in the tree being loaded, assuming it will replace them with data in the Gedcom file.
 +
 
 +
It is more careful with Places, since Places can contain data that cannot be (or is oftent not) loaded from Gedcom. Thus, when it purges Place records, it retains records that contain latitude or longitude values or descriptive notes. Still, it ignores placelevels and medialinks to Places, both of which cannot be loaded from Gedcom files.
 +
 
 +
Consequently, the Gedcom Import Purge mod modifies the Places purge at the start of the Gedcom Import process so that it retains:
 +
* All Places when there are multiple trees, but just one Place list,
 +
* Places with a longitude, latitude, note, or placelevel value, and
 +
* Places with a Medialink.
 +
<!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-places1 mw-customtoggle-places2" style="text-decoration:underline;color:#0645ad;text-align:right;">&#91;Hide details&#93;</div>
 +
</div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE -->
 +
 
 +
=== Purging Medialinks ===
 +
<!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-mlinks1 mw-customtoggle-mlinks2" id='mw-customcollapsible-mlinks1' style="text-decoration:underline;color:#0645ad;">&#91;Show Details&#93;</div>
 +
<div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-mlinks2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT -->
 +
<!-- BUTTON AT THE TOP OF TOGGLED CONTENT --><div class="mw-customtoggle-mlinks1 mw-customtoggle-mlinks2" style="padding-left:2em;text-decoration:underline;color:#0645ad;float:right;">&#91;Hide details&#93;</div>
 +
When the Gedcom Import Process purges data from the tree being loaded, it does not delete any Medialinks, presumably because
 +
* Many TNG admins don't count on GEDCOM to load (all) of their media items and links; they manually add other media items and links, and
 +
* Gedcom does not support medialinks for places or cemeteries, and any such links in the database must have been created manually, using the TNG editors
 +
However, as a result, Media links that have been deleted from the source database (that produced the Gedcom) will remain in TNG indefinitely, unless deleted manually
 +
 
 +
Consequently, the Gedcom Import Purge mod
 +
# Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process.
 +
# Modifies the Gedcom Import process to:
 +
## Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates and
 +
## Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - but only if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,
 +
# Modifies the Medialinks section of the Edit Media form to display and edit the "createdbygedcom" flag. The flag is turned off when new Medialinks are created via TNG data entry, and, from that point, can be changed, though it is not likely to be changed.
 +
<!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-mlinks1 mw-customtoggle-mlinks2" style="text-decoration:underline;color:#0645ad;text-align:right;">&#91;Hide details&#93;</div>
 +
</div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE -->
  
== Purpose of the Mod ==
+
=== Citation Medialinks ===
This mod changes the Gedcom Import process to
+
<!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-cm1 mw-customtoggle-cm2" id='mw-customcollapsible-cm1' style="text-decoration:underline;color:#0645ad;">&#91;Show Details&#93;</div>
# Add a flag (a new database field) to Medialinks that are created by Gedcom Imports,
+
<div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-cm2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT -->
# Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and
+
<!-- BUTTON AT THE TOP OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;float:right;">&#91;Hide details&#93;</div>
# Retain Places with Placelevels and/or Medialinks.
+
Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks.
(The Medialink purge can be prevented if the user unchecks a new "Purge Medialinks" checkbox on the Gedcom Import kickoff form.)
 
  
Without this mod, the Gedcom Import Process:
+
Now that the TNG Gedcom Import process does create citation medialinks, Gedcom files that have citation medialinks ''and'' higher-level medialinks generate redundant medialinks. In addition, (as of TNGv12.2) not all TNG program that display medialinks handle citation medialinks cleanly.
* Leaves ''all'' Medialinks intact, including those that have been removed from the source database, and
 
* May purge some Places that have Placelevels and/or Medialinks that cannot be replaced through the Gedcom Import.
 
  
=== Background ===
+
Consequently, the Gedcom Import Purge mod
When the "pristine" (unchanged by a mod) Gedcom Import process starts, and you tell it to replace "all current data", TNG deletes all People, Families, Events, Notes, Citations, Sources, and Repositories in that tree, because TNG assumes that they will all be replaced from data in the Gedcom file. But it does not delete any Media Links at all, presumably because
+
# Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process, and
* Many TNG admins don't count on Gedcom to load (all) of their media items and links, and instead manually create Media items and links, and 
+
# Modifies the Gedcom Import process to:
* Gedcom does not support Medialinks for places or cemeteries, and any such links in the database must have been created manually, using TNG data entry screens.
+
## Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates
 +
## Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,
 +
## Modify the Places purge so that it retains:
 +
##* All Places when there are multiple trees, but just one Place list,
 +
##* Places with a longitude, latitude, note, or placelevel value, or
 +
##* Places with a Medialink.
 +
## Ignore Citation Medialinks - if the user checks the new "Ignore Citation Medialinks" checkbox in the Gedcom Import kickoff form.
  
Also, the "pristine" Gedcom Import process purges Places that ''do not have'' geocodes or descriptive notes, presumably because it assumes that Places without such data will be replaced (as needed) by the data in the Gedcom file. However, it does not pay attention to the Placelevel field, or to Medialinks.
+
Some possible next steps:
 +
# The Gedcom Gedcom Converter mod should implement an option ''not to create'' those higher-level medialinks, and/or an option to remove the citation medialinks if it creates higher-level medialinks.
 +
# The Gedcom Import Purge mod should (perhaps) have an option to keep citation medialinks, and to ignore redundant higher-level medialinks.
 +
#* (Doing so might require the converter to look ahead in the Gedcom file to determine whether such redundant links exist, so it may not be practical to implement this feature.)
  
As a result, the pristine Gedcom Import process
+
Finally, note that it is common for Gedcom files to have direct links from media items to sources,people, and places, independently of citation medialinks. Thus, when I speak of suppressing or removing higher-level medialinks, I am referring to the redundant ones that duplicate a citation medialink, not to the direct medialinks.
# Retains Media links that have been deleted from the source database that produced the Gedcom, forcing the site Admin to delete those Medialinks manually (assuming that the Admin realizes that they exist).
+
<!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;text-align:right;">&#91;Hide details&#93;</div>
# Deletes Places that have Placelevels or Medialinks, but no geocodes or notes, thus deleting data that cannot be replaced by data in the Gedcom file.
+
</div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE -->
  
== Conflicts And Dependencies ==
 
No known conflicts. This mod counts on the presence of the [[Blue Info Button]] mod to format small help icons on the Gedcom Import kickoff screen, but [[Blue Info Button]] is not strictly required. (See the visualizations below.)
 
 
 
== Related Mods ==
 
== Related Mods ==
* [[Blue Info Button]] is not strictly required, but it is certainly recommended and has a very small footprint.
+
I do not know of any conflicts with other mods.
 +
* [[Admin Media Search]] - This mod changes the way that TNG displays all medialinks, and it handles citation medialinks, which are a significant concern of this mod.
 +
* [[Gedcom Import Mediatype]] and [[Gedcom Import Monitor]] are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are ''coordinated'' with this mod, but there are '''no dependencies'' among them and this mod.
 
* If the ''optional'' [[Show Mod Names]] mod is installed, this mod will utilize its functionality.  
 
* If the ''optional'' [[Show Mod Names]] mod is installed, this mod will utilize its functionality.  
* [[Gedcom Import Mediatype]] and [[Gedcom Import Monitor]] are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are ''coordinated'' with this mod (Gedcom Import Purge), but there are '''no dependencies'' among them and this mod.
 
  
==Requirements==
+
== Installation ==
* A working TNG installation.
+
=== Files Installed ===
* An installed current version of the [[Mod Manager]].
+
This mod has two mod subfolders:
* You should backup files listed in the panel on the right.
+
# '''gedcom_import_purge_v12.0.0.5''' - The subfolder that you expect to exist for any mod that installs files. It contains
 +
#*'''rrgedcomimportpurge_dbsetup.php''' This is ordinarily run once, from a link in the Mod Manager, to define the new Medialinks table field that keeps track of medialinks that are created by the Gedcom Import.  The field is called '''createdfromgedcom'''.
 +
# The shared folder '''RR-shared_mod_includes_v12.0.0.3''', which contains files that support mod option management in many of my mods:
 +
#* '''rrshared_innermodmenu2.php''' - [https://tng.lythgoes.net/wiki/index.php?title=Inner_Mod_Menus Inner&nbsp;Mod&nbsp;Menus]
 +
#* '''rrshared_modsettingsblocks2.php''' - [https://tng.lythgoes.net/wiki/index.php?title=Mod_Settings_Blocks Mod Settings Blocks]
 +
#* '''img/rrshared_wikilogo.gif''' - An icon that links to a mod's TNG Wiki page.
  
==Installation==
+
The shared folder is packaged with several mods, each of which may install some or all of its files. Once a file from the shared folder has been installed (by any mod), that file will not be overwritten or removed by the subsequent installation or un-installation of any mod (including this one), nor will its presence generate any Mod Manager errors.  
# Remove and delete previous version of this mod.
 
# Backup the files updated by this mod.  They are listed in the panel at the upper right.
 
# Download the .zip file, and extract its .cfg file '''and subfolder''' to the mods folder.
 
# Follow the normal automated installation for Mod Manager, as shown in the example [[Mod Manager - Installing Config Files]].
 
# '''You must also run the database setup program that is installed by this mod.  The only link to the database setup program is in this mod's ''Description'' field in the Mod Manager.''' The database setup program creates the database field that is used to flag Medialinks that are created by the Gedcom Import process.  If you fail to run that setup program, your next Gedcom Import will encounter a fatal database error.
 
  
== In the event of a problem ==
+
If you unzip mod distribution files directly into your mods folder, then the presence of this second subfolder and the installation of its files should be invisible to you.  But if you generally ''copy'' mod subfolders to your mods folder, you need to make sure to copy the folder '''RR-shared_mod_includes_v12.0.0.2''' (well, unless you are confident that it is already in your mods folder because it was part of another mod).
# Try using the [[Mod_Manager_-_Installing_Config_Files#Remove_Mod_Steps|Mod Manager Remove]] capability
+
{{RobinInstallationBoilerplate}}
# Contact me through [http://www.robinrichmond.com/family/mod_support.php My Mod Support form].
 
  
== Visualization of this Mod ==
+
== Visualizations ==
  
{| border="1" cellspacing="1" cellpadding="2" class="wikitable"
+
{| border="3" cellspacing="1" cellpadding="2" class="wikitable"
|-
 
! style="text-align:left;" | <span style="color: red">'''In the Mod Manager, After Installation'''</span>, showing the database setup hyperlink.
 
|-
 
| [[Image:gedcom_import_purge-modman.png]]
 
 
|-
 
|-
! style="text-align:left;" | You must follow the hyperlink shown above to run the setup program that creates the database field through which TNG remembers which media links were created by a Gedcom Import. <span style="color: red">The program that installs the new database field looks like this:</span>
+
| <span style="color: red">'''In the Mod Manager, After Installation'''</span>, showing the database setup hyperlink.
 +
<div>[[Image:gedcom_import_purge-modman.png]]</div>
 +
You must follow the hyperlink shown above to run the setup program that creates the database field through which TNG remembers which media links were created by a Gedcom Import.  
 
|-
 
|-
| [[Image:gedcom_import_purge-createdb.png]]
+
| <span style="color: red">The Database setup program</span>
 +
[[Image:gedcom_import_purge-createdb.png]]
 
|-  
 
|-  
! style="text-align:left;" | Admin >> Import/Export >> Import screen <span style="color: red">''' AFTER INSTALLATION:'''</span>
+
| <span style="color: red">'''The Gedcom Import Kickoff Form (Admin>>Import/Export) AFTER INSTALLATION:'''</span>
|-
+
<div>[[Image:gedcom_import_purge-after1.jpg]]</div>
| [[Image:gedcom_import_mods-after.png]]
+
New or changed elements on the Gedcom Import Kickoff form:
|-
+
# The Inner Mod Menu, which pulls down from the "Mod Information" heading, is shown in its open state.
| The visualization above shows 5 (well, really 6) installed mods:
+
# A mod option determines whether the "Import media if present" checkbox should be checked. For the Gedcom Import Purge mod to be useful in a Gedcom Import, this checkbox should be checked.
# Changes from Gedcom Import Purge (''this'' mod) are outlined in red.
+
# Two new checkboxes. The default status of each checkbox is determined by mod options.
# Changes from [[Gedcom Import Mediatype]] (which is independent of this mod) are outlined in orange.
+
## "Purge media links..." - The primary purpose of this mod.
# Changes from [[Gedcom Import Monitor]] (which is independent of this mod) are outlined in green.
+
## "Suppress the creation of citation medialinks" - It all depends on your Gedcom file and how you want to handle citation medialinks.
# The "Show Mod Names" button installed by the ''optional'' [[Show Mod Names]] mod is outlined in brown.  Most of my mods "register" themselves so that [[Show Mod Names]] can tell you (or, potentially, me) which of my mods affect the current program, and which version of each mod is installed. (If you choose to install [[Show Mod Names]], and use its default mod parameters, that button is visible only in Admin programs, not in any end-user programs.)
+
|}
# The ''optional but recommended'' [[Blue Info Button]] mod formats help links as shown in the visualization.  Without [[Blue Info Button]], the help links would just be underlined question marks.
+
 
# The visualization also is affected by the [[Gedcom Converter]] mod, which creates the "Gedcom Converter" tab up in the TNG Tab menu. But [[Gedcom Converter]] is otherwise unrelated to ''this'' mod (Gedcom Import Purge).
+
{| border="3" cellspacing="1" cellpadding="2" class="wikitable"
|-
+
| <span style="color: red;">'''Message As the Gedcom Import starts:'''</span>
! style="text-align:left;" | <span style="color: red">'''As the Gedcom Import starts:'''</span>
+
This message is visible only if you choose the "Old style import" checkbox (or, if you have installed [[Gedcom Import Monitor]], the "Scrolling progress listing").
|-
+
<div>[[Image:gedcom_import_purge-purge.png]]</div>
| [[Image:gedcom_import_purge-purge.png]]
+
Note that the Gedcom Import process will not purge any medialinks the ''first'' time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.
'''Note that the Gedcom Import process will not purge any medialinks the ''first'' time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.'''
+
|}
 +
 
 +
{| border="3" cellspacing="1" cellpadding="2" class="wikitable"
 +
| <span style="color: red">'''Media Links in the Admin>>Media>>Edit form'''</span>
 +
The Medialinks table in the Media Edit form now has a column with a checkbox for the "From Gedcom" flag. It is checked for Media items that were loaded through a Gedcom Import process, and defaults to unchecked (and usually remains that way) when Media items are added within TNG.
 +
<br>[[Image:gedcom_import_purge-after-medialinks.jpg]]
 +
|}
 +
 
 +
{| border="3" cellspacing="1" cellpadding="2" class="wikitable"
 +
| <span style="color: red">'''The Mod Options - in a Mod Settings Block'''</span>
 +
The Mod Settings Block for Gedcom Import Purge is at '''Admin>>Setup>>Import&nbsp;Settings'''. You can jump straight there from the "Change Mod Options" link in the Inner Mod Menu.<br>
 +
[[Image:gedcom_import_purge-after-modsettingsblock.jpg]]
 
|}
 
|}
  
== Mod Change History ==
+
== Revision History ==
 
{| border="0" cellspacing="1" cellpadding="2" class="wikitable"
 
{| border="0" cellspacing="1" cellpadding="2" class="wikitable"
|-
+
! Mod Version !! TNG Version !! Date !! Note
! Mod Version
+
|- style="vertical-align:top;"  <!-- ############################################################## -->
! TNG Versions
+
| [[Media:gedcom_import_purge_v12.0.0.5.zip|12.0.0.5]]
! Date
+
| 12.0+
! Note
+
| 2 Dec 2019
|-  
+
|
 +
# Modified the Place purge process so that it does not purge places on sites that have multiple trees and have one shared Places list.
 +
# Added an Inner Mod Menu, and moved the options to an Mod Settings Block
 +
# Added a form field in the Medialinks Admin>>Media>>Edit form so that admin users can set or unset the "createdfromgedcom" (aka "From Gedcom") flag.
 +
# Enhanced the message that, at the beginning of the Gedcom Import process, displays the counts of places and medialinks that were purged, and medialinks that were not purged.
 +
# The Gedcom Import process generates a list of the places that were purged and recreated, and purged and not recreated.
 +
# Added the "Suppress the creation of citation medialinks" checkbox on the Admin Gedcom Import kickoff form.
 +
# Added code to suppress citation medialinks when the checkbox is checked.
 +
# Adds a Save buttons to the left of the BodyText field in the Edit Media form, so that the user doesn't always have to scroll to the top or bottom of the form to finish.
 +
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 +
| [[Media:gedcom_import_purge_v12.0.0.4.zip|12.0.0.4]]
 +
| 12.0+
 +
| 15 May 2018
 +
| No functional changes; made compatible with TNGv12.
 +
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 +
| [[Media:textplus_charts_v10.1.0.3p.zip|10.1.0.3p]]
 +
| 10.1-11.1
 +
| 26 Mar 2017
 +
| Removed the second line from the cust_text.php target location search string
 +
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 
| 10.1.0.3
 
| 10.1.0.3
 
| 10.1-11.1
 
| 10.1-11.1
 
| 26 Mar 2017
 
| 26 Mar 2017
 
| A technical update that, mostly, just makes [[Show Mod Names]] optional, and avoids a installation conflict introduced by TNGv11.0.1. That is, this version omits the adminlib.php patch that was part of v2 of this mod, and changes this mod's database setup program so that it works independently of that adminlib.php patch (which was fixed in TNGv11.0.1).
 
| A technical update that, mostly, just makes [[Show Mod Names]] optional, and avoids a installation conflict introduced by TNGv11.0.1. That is, this version omits the adminlib.php patch that was part of v2 of this mod, and changes this mod's database setup program so that it works independently of that adminlib.php patch (which was fixed in TNGv11.0.1).
|-  
+
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 
| 10.1.0.2
 
| 10.1.0.2
 
| 10.1-11.0.1
 
| 10.1-11.0.1
 
| 25 May 2016
 
| 25 May 2016
 
| Removed a unneeded &lt;script> element that incorrectly referenced an external file, and that - under rare circumstances I still don't understand - could cause the database setup program to log you out from your TNG session.  Also fixes a similarly incorrect &lt;script> element in adminlib.php.
 
| Removed a unneeded &lt;script> element that incorrectly referenced an external file, and that - under rare circumstances I still don't understand - could cause the database setup program to log you out from your TNG session.  Also fixes a similarly incorrect &lt;script> element in adminlib.php.
|-  
+
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 
| 10.1.0.1b
 
| 10.1.0.1b
 
| 10.1-11.0
 
| 10.1-11.0
 
| 25 Mar 2016
 
| 25 Mar 2016
 
| Fixed an error in a JavaScript warning. Cleaned up the code. Changed the Mod Parameters to strings rather than boolean values to be more tolerant of data entry errors.
 
| Fixed an error in a JavaScript warning. Cleaned up the code. Changed the Mod Parameters to strings rather than boolean values to be more tolerant of data entry errors.
|-  
+
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 
| 10.1.0.1a
 
| 10.1.0.1a
 
| 10.1-10.1.3
 
| 10.1-10.1.3
 
| 21 Feb 2016
 
| 21 Feb 2016
 
| No new end-user functionality. Now depends on [[Show Mod Names|Show Mod Names v2+]].
 
| No new end-user functionality. Now depends on [[Show Mod Names|Show Mod Names v2+]].
|-  
+
 
 +
|- style="vertical-align:top;"  <!-- ############################################################## -->
 
| 10.1.0.1
 
| 10.1.0.1
 
| 10.1-10.1.3
 
| 10.1-10.1.3
Line 150: Line 231:
 
| 11.1.0
 
| 11.1.0
 
| English
 
| English
 +
|-
 +
| [http://genealogy.oceanwharf.com Hooley Family Links]
 +
| [[User:lastejas|Rick Hooley]]
 +
| Public/Private
 +
| [[User: lastejas|See Here]]
 +
| [[User: lastejas|See Here]]
 +
| EN
 +
|-
 +
| [https://SmeeFamily.com/ Pete's Research]
 +
| [[User:TBirdUK|TBirdUK]]
 +
| Public/Private
 +
| 12.0.0.5
 +
| 12.01+
 +
| English
 +
|-
 +
| [https://WokingFamily.com/ Woking Family Tree Project]
 +
| [[User:TBirdUK|TBirdUK]]
 +
| Public/Private
 +
| 12.0.0.5
 +
| 12.01+
 +
| English
 +
|-
 
|}
 
|}
  
 +
[[Category:Mods for TNG v12]]
 
[[Category:Mods for TNG v11]]
 
[[Category:Mods for TNG v11]]
 
[[Category:Mods for TNG v10]]
 
[[Category:Mods for TNG v10]]
Line 158: Line 262:
 
[[Category:gedimport_trees.php mods]]
 
[[Category:gedimport_trees.php mods]]
 
[[Category:gedimport_misc.php mods]]
 
[[Category:gedimport_misc.php mods]]
[[Category:cust_text.php English mods]]
+
[[Category:admin_editmedia.php mods]]
 +
[[Category:js/mediafind.js mods]]
 +
[[Category:js/mediautils.js mods]]
 +
[[Category:micro_medialinks.php mods]]
 +
[[Category:ajx_updateorder.php mods]]
 +
[[Category:admin_importconfig.php mods]]
 +
[[Category:admin_updateimportconfig mods]]
 +
[[Category:data_help.php mods]]
 +
[[Category:English cust_text.php mods]]

Revision as of 08:19, 18 December 2019

Gedcom Import Purge
Summary Causes the Gedcom Import Process to delete Medialinks records that have been deleted from the source database, and to retain some Places records that otherwise would be purged.
Validation
Mod Updated 2 Dec 2019
Download link v12.0.0.5.zip
TNG 12.0
Older versions can be downloaded from the Revision History.
Download stats
Author(s) Robin Richmond
Homepage this page
Mod Support My Mod Support form or TNG Community Forums
Contact Developer My Mod Support form
Latest Mod v12.0.0.5
Min TNG V 10.1
Max TNG V at least 12.2
Files modified
admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, admin_editmedia.php, js/mediafind.js, js/mediautils.js, micro_medialinks.php, ajx_updateorder.php, admin_importconfig.php, admin_updateimportconfig, English/data_help.php, English cust_text.php.


Installs: rradminbranches_ajx.php, rradminbranches_lib.php.


Installs shared files: rrshared_innermodmenu2.php, rrshared_modsettingsblocks2.php, img/rrshared_wikilogo.gif
Related Mods
Notes

Purpose of the Mod

This mod changes the Gedcom Import process to

  1. Add a flag (a new database field) to Medialinks that are created by Gedcom Imports,
  2. Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and
  3. Retain certain Places that are purged by the native code.

Without this mod, the Gedcom Import Process:

  • Leaves all Medialinks intact, including those that have been removed from the source database,
  • May purge Place records or Places data that is not replaced by the Gedcom Import.

Details

Purging Places

[Show Details]

When the "pristine" (unchanged by a mod) Gedcom Import process starts - and "all current data" is being replaced - it deletes all People, Families, Events, Notes, Citations, Sources, and Repositories in the tree being loaded, assuming it will replace them with data in the Gedcom file.

It is more careful with Places, since Places can contain data that cannot be (or is oftent not) loaded from Gedcom. Thus, when it purges Place records, it retains records that contain latitude or longitude values or descriptive notes. Still, it ignores placelevels and medialinks to Places, both of which cannot be loaded from Gedcom files.

Consequently, the Gedcom Import Purge mod modifies the Places purge at the start of the Gedcom Import process so that it retains:

  • All Places when there are multiple trees, but just one Place list,
  • Places with a longitude, latitude, note, or placelevel value, and
  • Places with a Medialink.
[Hide details]

Purging Medialinks

[Show Details]
[Hide details]

When the Gedcom Import Process purges data from the tree being loaded, it does not delete any Medialinks, presumably because

  • Many TNG admins don't count on GEDCOM to load (all) of their media items and links; they manually add other media items and links, and
  • Gedcom does not support medialinks for places or cemeteries, and any such links in the database must have been created manually, using the TNG editors

However, as a result, Media links that have been deleted from the source database (that produced the Gedcom) will remain in TNG indefinitely, unless deleted manually

Consequently, the Gedcom Import Purge mod

  1. Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process.
  2. Modifies the Gedcom Import process to:
    1. Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates and
    2. Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - but only if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,
  3. Modifies the Medialinks section of the Edit Media form to display and edit the "createdbygedcom" flag. The flag is turned off when new Medialinks are created via TNG data entry, and, from that point, can be changed, though it is not likely to be changed.
[Hide details]

Citation Medialinks

[Show Details]
[Hide details]

Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks.

Now that the TNG Gedcom Import process does create citation medialinks, Gedcom files that have citation medialinks and higher-level medialinks generate redundant medialinks. In addition, (as of TNGv12.2) not all TNG program that display medialinks handle citation medialinks cleanly.

Consequently, the Gedcom Import Purge mod

  1. Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process, and
  2. Modifies the Gedcom Import process to:
    1. Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates
    2. Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form,
    3. Modify the Places purge so that it retains:
      • All Places when there are multiple trees, but just one Place list,
      • Places with a longitude, latitude, note, or placelevel value, or
      • Places with a Medialink.
    4. Ignore Citation Medialinks - if the user checks the new "Ignore Citation Medialinks" checkbox in the Gedcom Import kickoff form.

Some possible next steps:

  1. The Gedcom Gedcom Converter mod should implement an option not to create those higher-level medialinks, and/or an option to remove the citation medialinks if it creates higher-level medialinks.
  2. The Gedcom Import Purge mod should (perhaps) have an option to keep citation medialinks, and to ignore redundant higher-level medialinks.
    • (Doing so might require the converter to look ahead in the Gedcom file to determine whether such redundant links exist, so it may not be practical to implement this feature.)

Finally, note that it is common for Gedcom files to have direct links from media items to sources,people, and places, independently of citation medialinks. Thus, when I speak of suppressing or removing higher-level medialinks, I am referring to the redundant ones that duplicate a citation medialink, not to the direct medialinks.

[Hide details]

Related Mods

I do not know of any conflicts with other mods.

  • Admin Media Search - This mod changes the way that TNG displays all medialinks, and it handles citation medialinks, which are a significant concern of this mod.
  • Gedcom Import Mediatype and Gedcom Import Monitor are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are coordinated with this mod, but there are 'no dependencies among them and this mod.
  • If the optional Show Mod Names mod is installed, this mod will utilize its functionality.

Installation

Files Installed

This mod has two mod subfolders:

  1. gedcom_import_purge_v12.0.0.5 - The subfolder that you expect to exist for any mod that installs files. It contains
    • rrgedcomimportpurge_dbsetup.php This is ordinarily run once, from a link in the Mod Manager, to define the new Medialinks table field that keeps track of medialinks that are created by the Gedcom Import. The field is called createdfromgedcom.
  2. The shared folder RR-shared_mod_includes_v12.0.0.3, which contains files that support mod option management in many of my mods:

The shared folder is packaged with several mods, each of which may install some or all of its files. Once a file from the shared folder has been installed (by any mod), that file will not be overwritten or removed by the subsequent installation or un-installation of any mod (including this one), nor will its presence generate any Mod Manager errors.

If you unzip mod distribution files directly into your mods folder, then the presence of this second subfolder and the installation of its files should be invisible to you. But if you generally copy mod subfolders to your mods folder, you need to make sure to copy the folder RR-shared_mod_includes_v12.0.0.2 (well, unless you are confident that it is already in your mods folder because it was part of another mod).

[Show Installation Details]

Requirements

  • A working TNG installation.
  • An installed current version of the Mod Manager.
  • You should backup files listed in the panel on the right.

Procedure

  1. Remove and delete previous version of this mod.
  2. Backup the files updated by this mod. They are listed in the panel at the upper right.
  3. Download the .zip file, Extract its .cfg file to the mods folder.
  4. Follow the normal automated installation for Mod Manager, as shown in the example Mod Manager - Installing Config Files.

Problems?

  1. Try using the Mod Manager Remove capability
  2. Contact me through My Mod Support form.
[Hide Details]

Visualizations

In the Mod Manager, After Installation, showing the database setup hyperlink.
Gedcom import purge-modman.png

You must follow the hyperlink shown above to run the setup program that creates the database field through which TNG remembers which media links were created by a Gedcom Import.

The Database setup program

Gedcom import purge-createdb.png

The Gedcom Import Kickoff Form (Admin>>Import/Export) AFTER INSTALLATION:
Gedcom import purge-after1.jpg

New or changed elements on the Gedcom Import Kickoff form:

  1. The Inner Mod Menu, which pulls down from the "Mod Information" heading, is shown in its open state.
  2. A mod option determines whether the "Import media if present" checkbox should be checked. For the Gedcom Import Purge mod to be useful in a Gedcom Import, this checkbox should be checked.
  3. Two new checkboxes. The default status of each checkbox is determined by mod options.
    1. "Purge media links..." - The primary purpose of this mod.
    2. "Suppress the creation of citation medialinks" - It all depends on your Gedcom file and how you want to handle citation medialinks.
Message As the Gedcom Import starts:

This message is visible only if you choose the "Old style import" checkbox (or, if you have installed Gedcom Import Monitor, the "Scrolling progress listing").

Gedcom import purge-purge.png

Note that the Gedcom Import process will not purge any medialinks the first time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.

Media Links in the Admin>>Media>>Edit form

The Medialinks table in the Media Edit form now has a column with a checkbox for the "From Gedcom" flag. It is checked for Media items that were loaded through a Gedcom Import process, and defaults to unchecked (and usually remains that way) when Media items are added within TNG.
Gedcom import purge-after-medialinks.jpg

The Mod Options - in a Mod Settings Block

The Mod Settings Block for Gedcom Import Purge is at Admin>>Setup>>Import Settings. You can jump straight there from the "Change Mod Options" link in the Inner Mod Menu.
Gedcom import purge-after-modsettingsblock.jpg

Revision History

Mod Version TNG Version Date Note
12.0.0.5 12.0+ 2 Dec 2019
  1. Modified the Place purge process so that it does not purge places on sites that have multiple trees and have one shared Places list.
  2. Added an Inner Mod Menu, and moved the options to an Mod Settings Block
  3. Added a form field in the Medialinks Admin>>Media>>Edit form so that admin users can set or unset the "createdfromgedcom" (aka "From Gedcom") flag.
  4. Enhanced the message that, at the beginning of the Gedcom Import process, displays the counts of places and medialinks that were purged, and medialinks that were not purged.
  5. The Gedcom Import process generates a list of the places that were purged and recreated, and purged and not recreated.
  6. Added the "Suppress the creation of citation medialinks" checkbox on the Admin Gedcom Import kickoff form.
  7. Added code to suppress citation medialinks when the checkbox is checked.
  8. Adds a Save buttons to the left of the BodyText field in the Edit Media form, so that the user doesn't always have to scroll to the top or bottom of the form to finish.
12.0.0.4 12.0+ 15 May 2018 No functional changes; made compatible with TNGv12.
10.1.0.3p 10.1-11.1 26 Mar 2017 Removed the second line from the cust_text.php target location search string
10.1.0.3 10.1-11.1 26 Mar 2017 A technical update that, mostly, just makes Show Mod Names optional, and avoids a installation conflict introduced by TNGv11.0.1. That is, this version omits the adminlib.php patch that was part of v2 of this mod, and changes this mod's database setup program so that it works independently of that adminlib.php patch (which was fixed in TNGv11.0.1).
10.1.0.2 10.1-11.0.1 25 May 2016 Removed a unneeded <script> element that incorrectly referenced an external file, and that - under rare circumstances I still don't understand - could cause the database setup program to log you out from your TNG session. Also fixes a similarly incorrect <script> element in adminlib.php.
10.1.0.1b 10.1-11.0 25 Mar 2016 Fixed an error in a JavaScript warning. Cleaned up the code. Changed the Mod Parameters to strings rather than boolean values to be more tolerant of data entry errors.
10.1.0.1a 10.1-10.1.3 21 Feb 2016 No new end-user functionality. Now depends on Show Mod Names v2+.
10.1.0.1 10.1-10.1.3 6 Feb 2016 New mod.

Sites using this mod

If you download and install this mod, please add your site to the table below.

URL User Note Mod-Version TNG-Version User-language
Robin Richmond's Genealogy Database - admin function; not visible. Robin Richmond Mod developer 10.1.0.3 11.1.0 English
Hooley Family Links Rick Hooley Public/Private See Here See Here EN
Pete's Research TBirdUK Public/Private 12.0.0.5 12.01+ English
Woking Family Tree Project TBirdUK Public/Private 12.0.0.5 12.01+ English