Gedcom Import Purge
Gedcom Import Purge | ||
---|---|---|
Summary | Causes the Gedcom Import Process to delete Medialinks records that have been deleted from the source database, and to retain some Places records that otherwise would be purged. | |
Validation | ||
Mod Updated | 23 Sep 2023 | |
Download link | v14.0.0.5h
(See the Revision History) | |
Download stats | ||
Author(s) | Robin Richmond | |
Homepage | this page | |
Mod Support | My Mod Support form or TNG Community Forums | |
Contact Developer | My Mod Support form | |
Latest Mod | 14.0.0.5h | |
Min TNG V | 10.1 | |
Max TNG V | 14.0.3 | |
Files modified | admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, admin_editmedia.php, js/mediafind.js, js/mediautils.js, micro_medialinks.php, ajx_updateorder.php, admin_importconfig.php, admin_updateimportconfig, English/data_help.php. | |
Related Mods | Gedcom Converter, Depends on Mod Settings Blocks | |
Notes |
Purpose of the Mod
This mod could have been called "Gedcom Import Medialinks" because its primary purpose is to make sure that old no-longer-valid imported Medialinks do not hang around and disrupt TNG pages after they have been deleted from the source database. To recognize imported Medialinks, it flags all Medialinks that are created through a Gedcom import. See the Purging Medialinks section below for details.
This mod also retains certain Places that are purged by the native TNG code.
This mod and its functionality are meaningful only when a Gedcom import replaces "All current data" in a tree. Data can be loaded from Gedcom files incrementally; that is, multiple Gedcom files can be loaded into one tree. But incremental Gedcom imports can make relationships among objects in a tree very difficult to define and maintain. Thus replacing "All current data" is the most straightforward and common way of doing Gedcom imports in TNG.
Significantly, to replace "All current data" is something of a misnomer. It does mean to replace all People, Families, Events, Notes, Citations, Sources, and Repositories, by purging all such records from the tree, and then replacing them with whatever is in the Gedcom file. (In this context, to "purge" a record basically means to "delete the record, with the expectation that the deleted records will be reloaded from the Gedcom file".)
But "All current data" excludes Media items and Medialinks, because
- Any media item can be linked to an object (a Person, Family, Event, Citation, Source, Repository, or Place) in a different tree,
- Medialinks to places cannot be defined in a Gedcom file. If they exist, they must have been created with TNG data entry forms,
- Some TNG admins define Media items and Medialinks independently of Gedcom imports, that is, strictly through TNG data entry forms, and
- (Without this mod), there is no way to distinguish between Medialinks created through TNG data entry forms, and Medialinks created through Gedcom Imports.
Details
Without this mod, when the Gedcom Import Process starts (and "All current data" is being replaced), it
- Purges all People, Families, Events, Notes, Citations, Sources, Repositories, and Citation Medialinks in the tree being loaded, assuming it will replace them with data in the Gedcom file,
- Leaves all Media items intact,
- Purges Place records that it believes do not contain data that cannot be replaced by Gedcom data, again, assuming that it can replace them with data in the Gedcom file, and
- Leaves all Medialinks, except Citation Medialinks, intact.
This mod leaves actions #1 and #2 just above as they are, tweaks #3, and significantly changes #4.
Purging Medialinks [Show Details]
The native Gedcom import process does purge Citation Medialinks, because it purges all Citation, at which point all Citation Medialinks become invalid. But it does not purge other Medialinks, because we know that some Media items and Medialinks that we want to keep' may not be loaded from the Gedcom file. For instance:
- Many TNG admins don't count on GEDCOM to load all ofdel their media items and links; they use TNG data entry forms to add some Media items and Medialinks, which cannot be replaced by Gedcom data.
- Gedcom does not support Medialinks to Places, so any such links in the database must have been created with the TNG data entry forms, and cannot be replaced by Gedcom data.
However, as a result, Media links that have been deleted from the source database that produced the Gedcom file will remain in TNG indefinitely, unless deleted manually
Consequently, the Gedcom Import Purge mod
- Installs a setup program that create a new database field in the Medialinks table to keep track of Medialinks that are created by the Gedcom Import process.
- Modifies the Gedcom Import process to:
- Use the new 'Imported from Gedcom" field to flag the Medialinks it creates and
- Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import,
- Modifies the Medialinks section of the Edit Media form to display and edit the 'Imported from Gedcom' flag.
- This mod creates a checkbox for each Medialink in a Media Edit page so that you can see whether a given mMdialink came from a Gedcom Import or from TNG data entry. The checkbox status can be changed, though you are not likely to want to change it.
Purging Places [Show Details]
The Gedcom Import is more careful with Places, since Places can contain data that cannot be (or is often not) loaded from Gedcom files. Thus, when the native Gedcom import purges Place records, it retains records that contain latitude or longitude values or descriptive notes. Still, it ignores placelevels and Medialinks to Places, both of which cannot be loaded from Gedcom files.
Consequently, the Gedcom Import Purge mod modifies the Places purge at the start of the Gedcom Import process so that it retains:
- All Places if there are multiple trees but just one Place list,
- Places with a longitude, latitude, note, or placelevel value, and
- Places that are pointed to by a Medialink.
Pre-Existing Medialinks [Show Details]
Starting with the second Gedcom import after Gedcom Import Purge is installed, the mod prevents the Gedcom Import process from leaving invalid Gedcom-import-created medialinks behind. It does so by purging all medialinks that are flagged as having been created by a Gedcom import. It is safe to purge old medialinks in this circumstance because
- Medialinks that were not created by a Gedcom import will not be purged, since thay have not been flagged, and
- All valid medialinks that were purged should be re-created by the Gedcom Import process.
The mod does not
- Remove invalid Medialinksthat already existed when Gedcom Import Purge was installed, nor
- Prevent Medialinks that were defined through TNG data entry screens from being invalidated by a Gedcom import.
Both of these conditions will be addressed in a future version of Gedcom Import Purge through a new "Secondary Process". Until that feature is added to the mod, you will have to use "manual" SQL queries to delete problematic Medialinks. You can execute some of these queries with the TNG reporting tool. Others will need to be executed through a separate program such as sqlMyAdmin.
- If you have never created any Medialinksthrough TNG data entry, you can confidently remove invalid medialinks by removing 'all' medialinks:
- If you have multiple trees, then use the SQL command
DELETE tng_medialinks WHERE gedcom='xx'
(where 'xx' is your tree abbreviation, - If you have only one tree, you can clean up your Medialinks table more thoroughly with the SQL command
TRUNCATE tng_medialinks
- If you have multiple trees, then use the SQL command
- If you created Medialinks to Places through TNG data entry, but have not used TNG data entry to define any other Medialinks, you can clean up your Medialinks table with one of these SQL commands:
- Before TNGv12:
DELETE tng_medialinks WHERE gedcom='xx' AND linktype!='L' AND linktype!=""
- IN TNGv12.0 and after:
DELETE tng_medialinks WHERE gedcom='xx' AND linktype!='L'
- Before TNGv12:
- If you have entered Medialink sother than Place Medialinks through TNG data entry, well, your options are limited. It is extremely difficult for to define SQL query that will remove invalid Medialinks while also preserving Medialinks defined through data entry. If that's what you need to do, then, practically speaking, you'll need to make note of the Medialinks that you entered manually, issue one of the SQL commands above, and then redefine those Medialinks after you do the first Gedcom Import after you install the mod. (At that point, you may want to define those medialinks in your source database rather than in Gedcom.)
Citation Medialinks [Show Details]
In TNG, a Citation Medialink is a link from a Media item to a Citation, and is defined in the Medialinks table. In Gedcom files, a Citation is a Source reference in an Event record inside a Person or Family Record. And a Citation Medialink is a Media item reference within a Source Citation. Thus, though TNG and Gedcom medialinks both define many-to-many relationships between Media items and Citations, those relations are defined in opposite directions in TNG and in Gedcom files.
When Family Tree Maker (FTM) Gedcom files contain media information, they define Citation Medialinks, and do not automatically link the related Media items to the People and Families in which the Citation Medialinks occur. But until version 12, TNG didn't support Citation Medialinks at all. That is, it did not import them, had no way to enter them through TNG data entry, and ignored them in Gedcom imports. Consequently - in the absence of a Gedcom converter - many Media items would not be represented in TNG Person Profiles or Family Charts. Thus, TNG Gedcom converters (all of which were initially created to handle invalid aspects) took note of Citation Medialinks, and created Person and Family medialinks to the Media items referenced by all of their Citation Medialinks.
Starting at some point within TNGv12 (not TNGv12.0, but an incremental version or two later), TNG's Gedcom Import did (and does) read Citation Medialinks from Gedcom file, and create its own Citation Medialinks. (In addition, TNG administrators can now define Citation Medialinks through the Media Links section of the Media Edit page, but that's not really relevant to this discussion.) Aside from the Media Edit page, Citation Medialinks are presented to users in two places:
- In the Person Profile, as hyperlinked thumbnail images in Source Citations,
- In the "Linked to" column of the end-user Media item search/results page, browsemedia.php, and
- In the "Linked to" column of the administrative Media item search/results page, admin_media.php.
Note that the native deletion of Citation Medialinks was implemented after TNG began importing Citation Medialinks. Thus, there was a short period of time in which old Citiation Medialinks did hang around and (sometimes) get in the way after a Gedcom Import. Since that problem no longer exists, there is not much point in one of the Gedcom Import Purge mod's 'features'; that is, the checkbox on the Gedcom Import kick-off form that allows the user to suppress the creation of Citation Medialinks. At this point, the only reason I can think of to suppress the creation of Citation Medialinks is to suppress the Media item thumbnails in the Sources section of the Person Profile. (And that's not very compelling.)
Other Mods
I do not know of any conflicts with other mods.
Related Mods
- The Mod Settings Blocks mod packages this mod's options into what I call a "Mod Settings Block", which is a set of form fields that formatted and labeled so that it stands out among other options in an Admin>>Setting form. Mod Settings Blocks is a prerequisite for this mod; it must be installed before this mod can be installed.
- An Inner Mod Menu, which provides handy links from TNG pages directly to mod documentation and a mod's Mods Settings Block. (Inner Mod Menus are visible only to TNG Admins; never to end users.)
- The Mods Used feature, which captures a list of mods that affected the code used to display a TNG page. (This is analogous to the Mod Manager Analyzer feature that reports which mods affect a given TNG file, but that applies at run-time.
- Gedcom Converter installs by the Gedcom converter program that creates (in the Gedcom file) the Person and Family Medialinks described above that are important when importing Gedcom files that do not have Person and Family Medialinks to the Media items associated with Citation Medialinks. (This applies to Family Tree Maker Gedcoms, and perhaps others.)
- Admin Media Search changes the way that TNG displays Medialinks, and it handles Citation Medialinks, which are a significant concern of this mod.
- Gedcom Import Mediatype is related only in that it also affects the Gedcom Import kickoff form and the Gedcom Import process. Aspects of that mods are coordinated with this mod, but there are no dependencies between them.
Installation
Files Installed
This mod does not install any files, but it does still have a mod subfolder, which contains the mod's language strings. That is, this mod's language strings are not defined in its .cfg file nor copied to cust_text.php files. As a result
- This mod's language strings are loaded only into the TNG programs that need them, not into all TNG programs, and
- It is easier to create, edit, test, and save translation for this mod than for mods that use standard cust_text.php strings.
This non-standard technique should be invisible to you unless you want to define a translation or override any of this mod's string values.
Read more about this technique, the simplified translation process, and the procedure for defining site-specific overrides to its language strings.
This mod installs one file from the .fg file.
rrgedcomimportpurge_dbsetup.php is the utility program that creates the database field that is used to flag Gedcom-Import-created medialinks. The fieldname is createdfromgedcom and it is in the Medialinks table. This utility program is ordinarily run once, from a link in the Mod Manager, to define that database field. If you no longer intend to use the mod, the utility program can be run a second time to remove the createdfromgedcom field from the database.
[Show Standard Installation Process]
Requirements
- A working TNG installation.
- An installed current version of the Mod Manager.
- You should backup files listed in the panel on the right.
Procedure
- Remove and delete previous version of this mod.
- Backup the files updated by this mod. They are listed in the panel at the upper right.
- Download the .zip file, Extract its .cfg file to the mods folder.
- Follow the normal automated installation for Mod Manager, as shown in the example Mod Manager - Installing Config Files.
Problems?
- Try using the Mod Manager Remove capability
- Contact me through My Mod Support form.
Visualizations
Message As the Gedcom Import starts: This message is visible only if you choose the "Old style import" checkbox (or, if you have installed Gedcom Import Monitor, the "Scrolling progress listing"). |
![]() |
Remember that the Gedcom Import process will not purge any medialinks the first time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import. |
Language Strings
As noted above, this mod's language strings are defined, distributed and files in its mod subfolder rather than in its .cfg file. TNG programs affected by this mod then load language strings from those mod subfolder files rather than from cust_text.php files. This non-standard technique for handling language strings has several advantages, including that
- It loads language strings only into the pages that need them rather than into all TNG pages, and
- It simplifies the process of defining and editing translations.
This non-standard technique should be invisible to you unless you want to define a translation or override any of this mod's string values.
Read more about this technique, the simplified translation process, and the procedure for defining site-specific overrides to its language strings.
Revision History
Mod Version | TNG Version | Date | Note |
---|---|---|---|
v14.0.0.5h | 14 | 23Sep2023 | Removed unneeded IMM code that Mod Settings Blocks 4 |
14.0.0.5g | 14.0 | 20Mar2023 | Upgrade to TNGv14 |
13.0.0.5e | 13.0-13.0.1 | 2Nov2020 | Tweaked on location's search text to accommodate a subtle change in TNGv13.0.1 |
13.0.0.5d | 13 | 31Oct2020 |
|
13.0.0.5c | 13 | 26Oct2020 | 5c Fixed language strings in the database-field-creation utility 5b Fixed language strings in the mod settings block |
12.0.0.5a | 12.0-12.3 | 20Aug2020 | Functional:
Visual & Internal:
|
12.0.0.5 | 12.0+ | 2Dec2019 |
|
12.0.0.4 | 12.0+ | 15May2018 | No functional changes; made compatible with TNGv12. |
10.1.0.13b | 10.1-11.1 | 26Mar2017 | Removed the second line from the cust_text.php target location search string |
10.1.0.3 | 10.1-11.1 | 26Mar2017 | A technical update that, mostly, just makes Show Mod Names optional, and avoids a installation conflict introduced by TNGv11.0.1. That is, this version omits the adminlib.php patch that was part of v2 of this mod, and changes this mod's database setup program so that it works independently of that adminlib.php patch (which was fixed in TNGv11.0.1). |
10.1.0.2 | 10.1-11.0.1 | 25May2016 | Removed a unneeded <script> element that incorrectly referenced an external file, and that - under rare circumstances I still don't understand - could cause the database setup program to log you out from your TNG session. Also fixes a similarly incorrect <script> element in adminlib.php. |
10.1.0.1b | 10.1-11.0 | 25Mar 2016 | Fixed an error in a JavaScript warning. Cleaned up the code. Changed the Mod Parameters to strings rather than boolean values to be more tolerant of data entry errors. |
10.1.0.1a | 10.1-10.1.3 | 21Feb2016 | No new end-user functionality. Now depends on Show Mod Names v2+. |
10.1.0.1 | 10.1-10.1.3 | 6Feb2016 | New mod. |
Sites using this mod
If you download and install this mod, please add your site to the table below.
URL | User | Note | Mod-Version | TNG-Version | User-language | |
---|---|---|---|---|---|---|
Robin Richmond's Genealogy Database - admin function; not visible. | Robin Richmond | Mod developer | 14.0.0.5h | 14.0 | English | |
Hooley Family Links | Rick Hooley | Public/Private | See Here | See Here | EN | |
Pete's Research | TBirdUK | Public/Private | 12.0.0.5 | 12.01+ | English | |
Woking Family Tree Project | TBirdUK | Public/Private | 12.0.0.5 | 12.01+ | English | |
KKs Family History | Kathy Kult | Public/Private | 14.0.0.5h |
|
English | |
Dabbling in Genealogy | Jeff Rueger | Public/Private | 13.0.0.5e | 13.1.2 | English |
- Mods for TNG v14
- Mods for TNG v13
- Mods for TNG v12
- Mods for TNG v11
- Mods for TNG v10
- Admin dataimport.php mods
- Admin gedimport.php mods
- Gedimport trees.php mods
- Gedimport misc.php mods
- Admin editmedia.php mods
- Js/mediafind.js mods
- Js/mediautils.js mods
- Micro medialinks.php mods
- Ajx updateorder.php mods
- Admin importconfig.php mods
- Admin updateimportconfig mods
- Data help.php mods