Line 1: |
Line 1: |
| {{TNGmod | | {{TNGmod |
| | mod_name = Gedcom Import Purge | | | mod_name = Gedcom Import Purge |
− | | mod_summary = Causes the Gedcom Import Process to delete Medialinks records that have been deleted in the source database, and to retain some Places records that otherwise would be purged. | + | | mod_summary = Causes the Gedcom Import Process to delete Medialinks records that have been deleted from the source database, and to retain some Places records that otherwise would be purged. |
| | mod_validation = | | | mod_validation = |
− | | mod_last_update = 15 May 2018 | + | | mod_last_update = 2 Dec 2019 |
− | | download_link = <div>[[Media:gedcom_import_purg_v12.0.0.4.zip|v12.0.0.4.zip]]{{Tv120}}</div> | + | | download_link = [[Media:gedcom_import_purge_v12.0.0.5.zip|v12.0.0.5.zip]]{{Tv120}} |
− | <div>[[Media:gedcom_import_purge_v10.1.0.3p.zip|gedcom_import_purge_v10.1.0.3p.zip]] {{Tv110}}{{Tv1010}}</div> | + | <div style='font-size:90%'>Older versions can be downloaded from the [[#Revision_History|Revision History]].</div> |
| | download_stats = | | | download_stats = |
| | mod_author = [[User:Robinrichm|Robin Richmond]] | | | mod_author = [[User:Robinrichm|Robin Richmond]] |
− | | mod_url = [http://www.robinrichmond.com/family/ Robin Richmond's Genealogy Database] | + | | mod_url = ''this page'' |
| | mod_contact = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] | | | mod_contact = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] |
| | mod_support = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] or [http://tng.community/ TNG Community Forums] | | | mod_support = [http://www.robinrichmond.com/family/mod_support.php My Mod Support form] or [http://tng.community/ TNG Community Forums] |
− | | mod_version = v10.1.0.3 & v12.0.0.4 | + | | mod_version = v12.0.0.5 |
| | min_TNG_ver = 10.1 | | | min_TNG_ver = 10.1 |
− | | max_TNG_ver = at least 12.0 | + | | max_TNG_ver = at least 12.2 |
− | | TNG_file_list = admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, English/data_help.php, English cust_text.php. installs rrgedcomimportpurge_dbsetup.php | + | | TNG_file_list = admin_dataimport.php, admin_gedimport.php, gedimport_trees.php, gedimport_misc.php, admin_editmedia.php, js/mediafind.js, js/mediautils.js, micro_medialinks.php, ajx_updateorder.php, admin_importconfig.php, admin_updateimportconfig, English/data_help.php, English cust_text.php. |
− | | related_mods = [[Blue Info Button]], [[Show Mod Names]], [[Gedcom Import Mediatype]], [[Gedcom Import Monitor]] (See "Related Mods" heading) | + | <br>Installs: rradminbranches_ajx.php, rradminbranches_lib.php. |
| + | <br>Installs shared files: rrshared_innermodmenu2.php, rrshared_modsettingsblocks2.php, img/rrshared_wikilogo.gif |
| + | | related_mods = [[Admin Media Search]], [[Gedcom Import Mediatype]], [[Gedcom Import Monitor]], [[Show Mod Names]] |
| | notes = | | | notes = |
| }} | | }} |
− |
| |
− | {| style="margin-right:0.5 em;" align="right"
| |
− | | __TOC__
| |
− |
| |
− | |}
| |
− |
| |
| == Purpose of the Mod == | | == Purpose of the Mod == |
| This mod changes the Gedcom Import process to | | This mod changes the Gedcom Import process to |
| # Add a flag (a new database field) to Medialinks that are created by Gedcom Imports, | | # Add a flag (a new database field) to Medialinks that are created by Gedcom Imports, |
| # Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and | | # Optionally, purge Medialinks that have that flag, but retains Medialinks without the flag (i.e. that were created through TNG data entry), and |
− | # Retain Places with Placelevels and/or Medialinks. | + | # Retain certain Places that are purged by the native code. |
− | (The Medialink purge can be prevented if the user unchecks a new "Purge Medialinks" checkbox on the Gedcom Import kickoff form.)
| + | {| style="margin-right:0.5 em;" align="right" |
− | | + | | __TOC__ |
| + | |} |
| Without this mod, the Gedcom Import Process: | | Without this mod, the Gedcom Import Process: |
− | * Leaves ''all'' Medialinks intact, including those that have been removed from the source database, and | + | * Leaves ''all'' Medialinks intact, including those that have been removed from the source database, |
− | * May purge some Places that have Placelevels and/or Medialinks that cannot be replaced through the Gedcom Import. | + | * May purge Place records or Places data that is not replaced by the Gedcom Import. |
| | | |
| === Background === | | === Background === |
Line 54: |
Line 51: |
| | | |
| === Citation Medialinks === | | === Citation Medialinks === |
− | (This section describes a problem that was introduced in TNGv12 that is not yet addressed by the this mod)
| + | This section describes how this mod addresses an an issue that was introduced in TNGv12, and how it could be addressed more effectively. |
− | | + | <!-- *** BEGIN DOUBLE TOGGLE --><div class="mw-collapsible mw-customtoggle-cm1 mw-customtoggle-cm2" id='mw-customcollapsible-cm1' style="text-decoration:underline;color:#0645ad;">[Show Details]</div> |
− | TNGv12 introduced Citation Medialinks, resulting in some unintended consequences. The TNG data entry process does not provide a way for users to link Citations to Media items, and, until TNGv12, the Gedcom Import ignored Citation Media. In most cases, ignoring Citation Media was OK because those media links (which are Gedcom OBJE references) were duplicated in higher-level records; that is, in the parent Event record and/or the Event's parent Person or Family record.
| + | <div class='mw-collapsible mw-collapsible-content mw-collapsed' id='mw-customcollapsible-cm2' style="border:thin solid grey;"><!-- BEGIN TOGGLED CONTENT --> |
− | | + | !-- BUTTON AT THE TOP OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;float:right;">[Hide details]</div> |
− | It is important to note that
| + | Until version 12, TNG's Gedcom Import process ignored Citation Medialinks, basically assuming (I presume) that the Gedcom medialinks were repeated in the relevant higher-level source and person or family records. In fact, because TNG did not handle citation medialinks, the "Modify Gedcom files for TNG" option in the Gedcom Converter mod specifically creates those higher-level medialinks when it encounters citation medialinks. |
− | * In many Genealogy applications, one citation can be applied to multiple events, event in multiple records. That is, a citation that describes the birth, death, and marriage of a husband and wife can be linked to the husband's name, birth, and death events, the wife's name, birth, and death events, plus their Family record's marriage event.
| |
− | * In contrast, in Gedcom, and in TNG, citations are strictly subordinate to a parent Event record. Consquently, the citation mentioned above sould become seven unique Citation records, each with a unique citationID. Also, if the citation is linked to a Media item, there would also be seven essentially-duplicate Citation Medialinks.
| |
− | | |
− | Citation Medialinks in TNG present several problems:
| |
− | # Since Citations do not have Gedcom IDs they get new CitationIDs in each Gedcom Import, and old Citation Medialinks survive a new Gedcom Import.
| |
− | #* Significantly, in Citation Medialinks, the citationID is the primary link destination. That is, unlike Event Medialinks, Citation Medialinks have the the citationID in the the field medialinks.personID. Citation Medialinks might not be retained, and might not present the other problems below if citationID was a separate field in the Medialinks table, making Citation medialinks subordinate to the Citation's parent eventID, and grandparent personID or FamilyID
| |
− | # TNGv12 also introduced Citation Medialinks in the Person Profile's Source Citation list. In many cases, this is a nice new feature, but
| |
− | #* The compiled list of Source Citations in the Person Profile normally merges identical Citations, and
| |
− | #* The Citation Medialink that has been added to the Person Profile Source Citations is represented by the unique medialinkID, not the MediaID. This makes all citations that have Medialinks unique, resulting in duplicate Source Citations.
| |
− | # For Media items that link to a Citation, the "Link" in Media search results is now dominated by Citation links. These search results represent each Medialink with the linked recordID. For Sources, People, Families, and Places, the recordID is generally meaningful to the user. But citationIDs do not carry meaning on their own, and cannot be looked up, so there is really no way to know what a citationID represents.
| |
− | #* This applies to the end-user media search program, browsemedia.php, and to the Admin search program admin_media.php.
| |
− | # When Media items are displayed (by showmedia.php), the web page tries to list all records that link to that media item - that is, as with Media search results, it lists the media item's medialinks. But unlike the search programs, showmedia.php turns the recordIDs into hyperlinks to that record. Well, since there is no way to link to a Citation, citationIDs sometime (but evidently not always) break that list of links.
| |
− | #* (I have not figured out exactly why this occurs with some media items and not others.)
| |
| | | |
− | '''Possible solutions:''' | + | Now that the TNG Gedcom Import process does create citation medialinks, Gedcom files that have citation medialinks ''and'' higher-level medialinks generate redundant medialinks. In addition, (as of TNGv12.2) not all TNG program that display medialinks handle citation medialinks cleanly. |
− | Partial solutions include:
| |
− | * Manually purge Citation Medialinks before a Gedcom Import with the simple SQL query
| |
− | *: DELETE FROM tng_medialinks WHERE linktype=""
| |
− | * This mod could purge all Citation Medialinks before each Gedcom Import,
| |
− | * Patch the getCitations function in personlib.php to remove linked media items from citations, or perhaps to change the link's ID from the medialinkID to the mediaID.
| |
− | * Patch showmedia.php to handle Citation Medialinks
| |
− | * Patch browsemedia.php and admin_media.php to sort Citation Medialinks to the end of the links, or perhaps to omit Citation Medialinks.
| |
| | | |
− | More comprehensive workarounds:
| + | Consequently, the Gedcom Import Purge mod |
− | * After a Gedcom Import, remove all Citation Medialinks with the SQL query shown above
| + | # Installs a setup program that create a new database field named "createdbygedcom" for keeping track of Medialinks that are created by the Gedcom Import Process, and |
− | * Modify the Gedcom Converter mod to remove Citation Media references when the parent or grandparent events have the same Media reference, or to remove Citation Media altogether.
| + | # Modifies the Gedcom Import process to: |
| + | ## Use the new "createdbygedcom" Medialinks field to flag the Medialinks it creates |
| + | ## Purge (delete) Medialinks that are flagged as having been created by a Gedcom Import - if the user checks the new "Purge media links created by previous Gedcom imports" checkbox in the Gedcom Import kickoff form, |
| + | ## Modify the Places purge so that it retains: |
| + | ##* All Places when there are multiple trees, but just one Place list, |
| + | ##* Places with a longitude, latitude, note, or placelevel value, or |
| + | ##* Places with a Medialink. |
| + | ## Ignore Citation Medialinks - if the user checks the new "Ignore Citation Medialinks" checkbox in the Gedcom Import kickoff form. |
| | | |
− | The ultimate solution, perhaps: | + | Some possible next steps: |
− | * Remove the new TNG Gedcom Import code that creates Medialinks for Citation Media. | + | # The Gedcom Gedcom Converter mod should implement an option ''not to create'' those higher-level medialinks, and/or an option to remove the citation medialinks if it creates higher-level medialinks. |
| + | # The Gedcom Import Purge mod should (perhaps) have an option to keep citation medialinks, and to ignore redundant higher-level medialinks. |
| + | #* (Doing so might require the converter to look ahead in the Gedcom file to determine whether such redundant links exist, so it may not be practical to implement this feature.) |
| | | |
− | == Conflicts And Dependencies ==
| + | Finally, note that it is common for Gedcom files to have direct links from media items to sources,people, and places, independently of citation medialinks. Thus, when I speak of suppressing or removing higher-level medialinks, I am referring to the redundant ones that duplicate a citation medialink, not to the direct medialinks. |
− | No known conflicts. This mod counts on the presence of the [[Blue Info Button]] mod to format small help icons on the Gedcom Import kickoff screen, but [[Blue Info Button]] is not strictly required. (See the visualizations below.)
| + | <!-- BUTTON AT THE BOTTOM OF TOGGLED CONTENT --><div class="mw-customtoggle-cm1 mw-customtoggle-cm2" style="text-decoration:underline;color:#0645ad;text-align:right;">[Hide details]</div> |
| + | </div><!-- END TOGGLED CONTENT and END DOUBLE TOGGLE --> |
| | | |
| == Related Mods == | | == Related Mods == |
| + | I do not know of any conflicts with other mods. |
| + | * [[Admin Media Search]] - This mod changes the way that TNG displays all medialinks, and it handles citation medialinks, which are a significant concern of this mod. |
| + | * [[Gedcom Import Mediatype]] and [[Gedcom Import Monitor]] are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are ''coordinated'' with this mod, but there are '''no dependencies'' among them and this mod. |
| * If the ''optional'' [[Show Mod Names]] mod is installed, this mod will utilize its functionality. | | * If the ''optional'' [[Show Mod Names]] mod is installed, this mod will utilize its functionality. |
− | * [[Gedcom Import Mediatype]] and [[Gedcom Import Monitor]] are related only in that they also affect the Gedcom Import kickoff form and the Gedcom Import process. Aspects of those mods are ''coordinated'' with this mod (Gedcom Import Purge), but there are '''no dependencies'' among them and this mod.
| |
| | | |
− | ==Requirements== | + | == Installation == |
− | * A working TNG installation.
| + | === Files Installed === |
− | * An installed current version of the [[Mod Manager]]. | + | This mod has two mod subfolders: |
− | * You should backup files listed in the panel on the right. | + | # '''gedcom_import_purge_v12.0.0.5''' - The subfolder that you expect to exist for any mod that installs files. It contains |
| + | #*'''rrgedcomimportpurge_dbsetup.php''' This is ordinarily run once, from a link in the Mod Manager, to define the new Medialinks table field that keeps track of medialinks that are created by the Gedcom Import. The field is called '''createdfromgedcom'''. |
| + | # The shared folder '''RR-shared_mod_includes_v12.0.0.3''', which contains files that support mod option management in many of my mods: |
| + | #* '''rrshared_innermodmenu2.php''' - [https://tng.lythgoes.net/wiki/index.php?title=Inner_Mod_Menus Inner Mod Menus] |
| + | #* '''rrshared_modsettingsblocks2.php''' - [https://tng.lythgoes.net/wiki/index.php?title=Mod_Settings_Blocks Mod Settings Blocks] |
| + | #* '''img/rrshared_wikilogo.gif''' - An icon that links to a mod's TNG Wiki page. |
| | | |
− | ==Installation==
| + | The shared folder is packaged with several mods, each of which may install some or all of its files. Once a file from the shared folder has been installed (by any mod), that file will not be overwritten or removed by the subsequent installation or un-installation of any mod (including this one), nor will its presence generate any Mod Manager errors. |
− | # Remove and delete previous version of this mod.
| |
− | # Backup the files updated by this mod. They are listed in the panel at the upper right.
| |
− | # Download the .zip file, and extract its .cfg file '''and subfolder''' to the mods folder.
| |
− | # Follow the normal automated installation for Mod Manager, as shown in the example [[Mod Manager - Installing Config Files]].
| |
− | # '''You must also run the database setup program that is installed by this mod. The only link to the database setup program is in this mod's ''Description'' field in the Mod Manager.''' The database setup program creates the database field that is used to flag Medialinks that are created by the Gedcom Import process. If you fail to run that setup program, your next Gedcom Import will encounter a fatal database error.
| |
| | | |
− | == In the event of a problem ==
| + | If you unzip mod distribution files directly into your mods folder, then the presence of this second subfolder and the installation of its files should be invisible to you. But if you generally ''copy'' mod subfolders to your mods folder, you need to make sure to copy the folder '''RR-shared_mod_includes_v12.0.0.2''' (well, unless you are confident that it is already in your mods folder because it was part of another mod). |
− | # Try using the [[Mod_Manager_-_Installing_Config_Files#Remove_Mod_Steps|Mod Manager Remove]] capability
| + | {{RobinInstallationBoilerplate}} |
− | # Contact me through [http://www.robinrichmond.com/family/mod_support.php My Mod Support form].
| |
| | | |
− | == Visualization of this Mod == | + | == Visualizations == |
| | | |
| {| border="1" cellspacing="1" cellpadding="2" class="wikitable" | | {| border="1" cellspacing="1" cellpadding="2" class="wikitable" |
Line 126: |
Line 114: |
| ! style="text-align:left;" | Admin >> Import/Export >> Import screen <span style="color: red">''' AFTER INSTALLATION:'''</span> | | ! style="text-align:left;" | Admin >> Import/Export >> Import screen <span style="color: red">''' AFTER INSTALLATION:'''</span> |
| |- | | |- |
− | | [[Image:gedcom_import_mods-after.png]] | + | | [[Image:gedcom_import_purge-after1.jpg]] |
| |- | | |- |
− | | The visualization above shows 5 (well, really 6) installed mods: | + | | New or changed elements on the Gedcom Import Kickoff form: |
− | # Changes from Gedcom Import Purge (''this'' mod) are outlined in red. | + | # The Inner Mod Menu, which pulls down from the "Mod Information" heading, is shown in its open state. |
− | # Changes from [[Gedcom Import Mediatype]] (which is independent of this mod) are outlined in orange. | + | # A mod option determines whether the "Import media if present" checkbox should be checked. For the Gedcom Import Purge mod to be useful in a Gedcom Import, this checkbox should be checked. |
− | # Changes from [[Gedcom Import Monitor]] (which is independent of this mod) are outlined in green.
| + | # Two new checkboxes. The default status of each checkbox is determined by mod options. |
− | # The "Show Mod Names" button installed by the ''optional'' [[Show Mod Names]] mod is outlined in brown. Most of my mods "register" themselves so that [[Show Mod Names]] can tell you (or, potentially, me) which of my mods affect the current program, and which version of each mod is installed. (If you choose to install [[Show Mod Names]], and use its default mod parameters, that button is visible only in Admin programs, not in any end-user programs.) | + | ## "Purge media links..." - The primary purpose of this mod. |
− | # The ''optional but recommended'' [[Blue Info Button]] mod formats help links as shown in the visualization. Without [[Blue Info Button]], the help links would just be underlined question marks. | + | ## "Suppress the creation of citation medialinks" - It all depends on your Gedcom file and how you want to handle citation medialinks. |
− | # The visualization also is affected by the [[Gedcom Converter]] mod, which creates the "Gedcom Converter" tab up in the TNG Tab menu. But [[Gedcom Converter]] is otherwise unrelated to ''this'' mod (Gedcom Import Purge). | |
| |- | | |- |
| ! style="text-align:left;" | <span style="color: red">'''As the Gedcom Import starts:'''</span> | | ! style="text-align:left;" | <span style="color: red">'''As the Gedcom Import starts:'''</span> |
| |- | | |- |
− | | [[Image:gedcom_import_purge-purge.png]] | + | | This message is visible only if you choose the "Old style import" checkbox (or, if you have installed [[Gedcom Import Monitor]], "Scrolling progress listing"). |
| + | <div> [[Image:gedcom_import_purge-purge.png]]</div> |
| '''Note that the Gedcom Import process will not purge any medialinks the ''first'' time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.''' | | '''Note that the Gedcom Import process will not purge any medialinks the ''first'' time you run it after installation, since the new database field hasn't yet been populated by flags that say that the Medialink was created by a Gedcom Import.''' |
| |} | | |} |