Lossless data migration: what the term requires – and where system change fails

In many companies, a system change is imminent in the coming months – ERP replacement, shutdown of a historically grown in-house development, move to the cloud. The decision for the new system is rarely the problem. The stumbling block almost always becomes what comes after: the data. Anyone who specifies “lossless data migration” usually means something different from what the term technically implies. This article clarifies what is actually behind it, what projects fail and what you have to realistically expect. How we implement this in practice is on our side Data Migration.

Why there are so many system changes now

The most striking driver is a date: the end of 2027, the mainstream maintenance for SAP ECC 6.0 expires, after which there is only paid Extended Maintenance. The Computerwoche reported, citing Gartner, that by the end of 2024 only about 14,000 of 35,000 ECC customers had migrated – and that 40 to 45 percent are expected to remain with the old system beyond 2027.

For SMEs, this is less an SAP question than a symptom. The same mechanics applies to Windows and database versions that fall out of support, to CRM systems whose manufacturers switch to a pure cloud model, and to individual software that no one can wait anymore, because the developers from back then work elsewhere. The trigger is different, the actual work is the same every time – and it lies in the data.

The selection of the target system takes months and binds the management. Data migration is often dealt with in a subset of the specifications. Conversely, project risk is distributed.

What “lossless” technically really means

Colloquially means lossless: nothing is lost. This is worthless as a requirement because it cannot be verified. The term only becomes reliable if it is divided into three criteria.

Completeness it is the most obvious and simple. Every data set that is technically relevant in the old system exists after the migration in the target system. Testable via volume skeletons and sum reconciliations – if there are 84.312 documents in the old system and 84.107 in the target system, the difference must be explainable, sentence by sentence.

Reference and semantic integrity it's the part that usually gets stuck. An invoice without associated customers is formally migrated and technically garbage. Still more difficult are silent shifts in meaning: a status field with five forms in the old system meets a target system that only knows three. Technically, it can be mapped. Whether the evaluation says the same afterwards, not the technology, but the specialist department decides.

Traceability means that for each data set in the target system, it is documented where it came from and which transformation it has undergone. This is not a diligent task for the project files, but the prerequisite for being able to react sensibly to errors at all – and simply a duty in regulated areas.

What migrations fail in practice

The error patterns are repeated with remarkable reliability:

• Data quality is checked too late. Duplicates, Karteilichen and inconsistent spellings arise over years and are not noticeable in day-to-day business, because the processing corrects them in the head. The target system cannot do that. Anyone who only finds out during the test run that a quarter of the customer master data is unusable has already lost the schedule.
• The mapping remains incomplete. Fields without correspondence in the target system are gladly postponed – and then end up in the free text field or nowhere. Each unmapped field is a conscious decision or subsequent data loss; There is no third.
• There is no real test run. A migration with 500 test data sets says nothing about runtimes, locks and storage behavior at 5 million. Only the test migration to full stock, with subsequent adjustment against the old system, is resilient.
• The way back is not planned. If it becomes clear on Monday morning that it is not running, the question of whether you can return to the old system within a few hours, decides on the level of damage and nerves. You want to know this answer before the go-live.

How long does a data migration take?

The honest answer: This cannot be said seriously without looking at two variables – the number of data objects to be migrated and the accessibility of the old system. A cleanly documented source system with an open database behaves completely differently than an application from which data can only be obtained via report exports.

What can be said reliably, however, is the distribution of effort. Analysis, mapping and cleanup regularly require a multiple of the actual transfer. The script that writes the data is built in days; Clarifying what “active customer” means in both systems takes weeks and requires the specialist department. Projects are almost never delayed because the technology was too slow, but because technical decisions were not made in time.

A sample that has proven itself: The cleanup starts in parallel with the selection of the target system, not afterwards. Duplicates can be eliminated in the old system long before it is clear where it is going – and this work is not in vain in every scenario.

What determines the price

Flat-rate prices for data migrations should be avoided, because they presuppose that someone already knows the state of the data. This is never the case before the analysis. The costs drive essentially four factors: the number of different data objects – not the amount of data, because ten million documents of a type are simpler than fifty different object types with a thousand sets each; the status of the old data; the question of how much history actually has to be included; and the number of connected systems whose interfaces also want to be adapted.

The most effective cost leverage lies in the last point. The question “Do we really need twelve years of movement data in the new system?” is uncomfortable because nobody wants to answer it alone. Nevertheless, it is more rewarding than any technical optimization – history that moves into an archive instead of into the production system costs a fraction.

How we proceed

Our process follows what results from the error images above. At the beginning there is AnalysisRecording the structure and quality of the source data, contrasting the target structure, fully documenting mapping – including the conscious decision which fields do not come along. The following is the Cleaning with duplicate matching, normalization and enrichment, the result of which must be measurable and not claimed. And finally Transfer via at least one test migration to full stock with subsequent 1:1 matching before the controlled go-live with a defined return route takes place.

After that, the real benefit begins: Only on consolidated data can Evaluations and Dashboards which the department also believes. If the system change is related to a move to the cloud, this is closely linked to the Cloud Migration into each other – the order of cleaning and moving then wants to be consciously determined.

Conclusion

Lossless data migration is not a tool you buy, but a proof you keep: completeness, integrity and traceability, proven by a comparison against the old system. The biggest leverage lies earlier than most project plans – in the data cleanup that can be started before the target system is even established. If you have a system change and you want to realistically assess the state of your data, talk to us.

Additional sources

• Computer Week – S/4HANA migration stalls: Many customers remain loyal to ECC even after 2027
• SAP – maintenance strategy and switch to S/4HANA