{"id":37,"date":"2023-01-18T16:06:04","date_gmt":"2023-01-18T16:06:04","guid":{"rendered":"https:\/\/deguzman.ph\/?p=37"},"modified":"2025-10-16T01:29:34","modified_gmt":"2025-10-16T01:29:34","slug":"evaluating-digital-reform-outcomes-by-roberto-de-guzman","status":"publish","type":"post","link":"https:\/\/deguzman.ph\/index.php\/2023\/01\/18\/evaluating-digital-reform-outcomes-by-roberto-de-guzman\/","title":{"rendered":"Don\u2019t Just Build Systems. Build Change: Rethinking Digital Reform Evaluation"},"content":{"rendered":"<p data-start=\"313\" data-end=\"662\">Digital transformation is often celebrated as a sign of progress \u2014 faster services, better transparency, and improved efficiency. But simply launching a new platform or digitizing processes doesn\u2019t guarantee success. Too often, reform efforts are declared \u201ccomplete\u201d once a system goes live, even when the institution behind it hasn\u2019t truly changed.<\/p>\n<p data-start=\"664\" data-end=\"794\">The real question isn\u2019t whether a system works.<br data-start=\"711\" data-end=\"714\" \/>It\u2019s whether the <strong data-start=\"731\" data-end=\"794\">organization and its people work differently because of it.<\/strong><\/p>\n<h4 data-start=\"801\" data-end=\"846\">The Myth of \u201cCompletion\u201d in Digital Reform<\/h4>\n<p data-start=\"848\" data-end=\"1162\">One of the most common misconceptions is that digital reform is a project with a finish line. A government rolls out an online system, trains users, or automates a process \u2014 and the reform is marked as \u201cdone.\u201d But transformation is not a one-time deployment; it\u2019s an ongoing shift in behavior, capacity, and trust.<\/p>\n<p data-start=\"1164\" data-end=\"1197\">A meaningful evaluation must ask:<\/p>\n<ul data-start=\"1198\" data-end=\"1425\">\n<li data-start=\"1198\" data-end=\"1257\">\n<p data-start=\"1200\" data-end=\"1257\"><strong data-start=\"1200\" data-end=\"1255\">Has trust improved among citizens using the system?<\/strong><\/p>\n<\/li>\n<li data-start=\"1258\" data-end=\"1340\">\n<p data-start=\"1260\" data-end=\"1340\"><strong data-start=\"1260\" data-end=\"1338\">Have staff behaviors adapted, or are old manual workarounds still in play?<\/strong><\/p>\n<\/li>\n<li data-start=\"1341\" data-end=\"1425\">\n<p data-start=\"1343\" data-end=\"1425\"><strong data-start=\"1343\" data-end=\"1425\">Is the institution better equipped to learn, respond, and adapt going forward?<\/strong><\/p>\n<\/li>\n<\/ul>\n<p data-start=\"1427\" data-end=\"1509\">If the answers to these questions are unclear, the reform is still only skin-deep.<\/p>\n<h4 data-start=\"1516\" data-end=\"1559\">Beyond KPIs: The Politics of Measurement<\/h4>\n<p data-start=\"1561\" data-end=\"1726\">Key performance indicators (KPIs) are helpful, but they can be misleading. Counting logins, transactions, or uptime might show activity \u2014 but not necessarily impact.<\/p>\n<p data-start=\"1728\" data-end=\"1976\">A surge in digital transactions could mean success \u2014 or it could mean users have no alternative, even if they\u2019re dissatisfied. Metrics alone don\u2019t reveal if access is equitable, if processes are fairer, or if bottlenecks have simply been digitized.<\/p>\n<p data-start=\"1978\" data-end=\"2036\"><strong data-start=\"1978\" data-end=\"2034\">True evaluation requires understanding power shifts:<\/strong><\/p>\n<ul data-start=\"2037\" data-end=\"2189\">\n<li data-start=\"2037\" data-end=\"2070\">\n<p data-start=\"2039\" data-end=\"2070\">Who benefits from the reform?<\/p>\n<\/li>\n<li data-start=\"2071\" data-end=\"2097\">\n<p data-start=\"2073\" data-end=\"2097\">Who has been left out?<\/p>\n<\/li>\n<li data-start=\"2098\" data-end=\"2189\">\n<p data-start=\"2100\" data-end=\"2189\">Has the technology distributed accountability \u2014 or reinforced the same rigid hierarchies?<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"2191\" data-end=\"2269\">Reform isn\u2019t just technical. It\u2019s political. And evaluation must reflect that.<\/p>\n<h4 data-start=\"2276\" data-end=\"2322\">Behavior Change: The Real Marker of Success<\/h4>\n<p data-start=\"2324\" data-end=\"2401\">A digital reform is only as strong as the <strong data-start=\"2366\" data-end=\"2387\">behavioral shifts<\/strong> it creates.<\/p>\n<ul data-start=\"2402\" data-end=\"2613\">\n<li data-start=\"2402\" data-end=\"2478\">\n<p data-start=\"2404\" data-end=\"2478\">Are public servants making decisions based on new, data-driven insights?<\/p>\n<\/li>\n<li data-start=\"2479\" data-end=\"2535\">\n<p data-start=\"2481\" data-end=\"2535\">Have citizens changed how they engage with services?<\/p>\n<\/li>\n<li data-start=\"2536\" data-end=\"2613\">\n<p data-start=\"2538\" data-end=\"2613\">Are outdated practices fading, or do they still lurk behind the new system?<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"2538\" data-end=\"2613\"><em data-start=\"2617\" data-end=\"2627\">Example:<\/em> A city may digitize business permits, but if officials still demand printed copies \u201cjust in case,\u201d the transformation is incomplete.<\/p>\n<p data-start=\"2762\" data-end=\"2916\">Measuring these changes requires blending behavioral diagnostics, user feedback, and cultural analysis into evaluation \u2014 not just system performance data.<\/p>\n<h4 data-start=\"2923\" data-end=\"2964\">Institutional Memory: Will It Survive?<\/h4>\n<p data-start=\"2966\" data-end=\"3176\">Another overlooked factor is <strong data-start=\"2995\" data-end=\"3020\">institutional memory.<\/strong> Many reforms collapse when key people move on or when they are overly dependent on external consultants. To ensure sustainability, evaluation must examine:<\/p>\n<ul data-start=\"3177\" data-end=\"3363\">\n<li data-start=\"3177\" data-end=\"3230\">\n<p data-start=\"3179\" data-end=\"3230\">Has knowledge been transferred to in-house teams?<\/p>\n<\/li>\n<li data-start=\"3231\" data-end=\"3291\">\n<p data-start=\"3233\" data-end=\"3291\">Are there internal champions who can drive improvements?<\/p>\n<\/li>\n<li data-start=\"3292\" data-end=\"3363\">\n<p data-start=\"3294\" data-end=\"3363\">Can the institution build on the reform without outside intervention?<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"3365\" data-end=\"3467\">If these answers are \u201cno,\u201d then what\u2019s been achieved is not transformation \u2014 it\u2019s a temporary upgrade.<\/p>\n<h4 data-start=\"3474\" data-end=\"3510\">Smarter Evaluation: Mixed Methods<\/h4>\n<p data-start=\"3512\" data-end=\"3556\">The best digital reform evaluations combine:<\/p>\n<ul data-start=\"3557\" data-end=\"3721\">\n<li data-start=\"3557\" data-end=\"3637\">\n<p data-start=\"3559\" data-end=\"3637\"><strong data-start=\"3559\" data-end=\"3583\">Quantitative metrics<\/strong> (system uptime, costs saved, speed of transactions)<\/p>\n<\/li>\n<li data-start=\"3638\" data-end=\"3721\">\n<p data-start=\"3640\" data-end=\"3721\"><strong data-start=\"3640\" data-end=\"3664\">Qualitative insights<\/strong> (interviews, frontline feedback, user experience audits)<\/p>\n<\/li>\n<\/ul>\n<blockquote data-start=\"3723\" data-end=\"3866\">\n<p data-start=\"3725\" data-end=\"3866\">A dashboard may show 95% uptime, but interviews might reveal that rural users can\u2019t access the system due to connectivity or lack of support.<\/p>\n<\/blockquote>\n<p data-start=\"3868\" data-end=\"3993\">By combining numbers and narratives, leaders get a true picture of reform performance \u2014 and can adapt before failure sets in.<\/p>\n<h4 data-start=\"4000\" data-end=\"4033\">From Reports to Learning Loops<\/h4>\n<p data-start=\"4035\" data-end=\"4194\">Most evaluations are post-mortems: written too late, read by few, and rarely acted upon. Instead, evaluation should function as a <strong data-start=\"4165\" data-end=\"4194\">continuous learning loop:<\/strong><\/p>\n<ul data-start=\"4195\" data-end=\"4378\">\n<li data-start=\"4195\" data-end=\"4253\">\n<p data-start=\"4197\" data-end=\"4253\">Embed reflection checkpoints within each project phase<\/p>\n<\/li>\n<li data-start=\"4254\" data-end=\"4313\">\n<p data-start=\"4256\" data-end=\"4313\">Regularly collect feedback from both staff and citizens<\/p>\n<\/li>\n<li data-start=\"4314\" data-end=\"4378\">\n<p data-start=\"4316\" data-end=\"4378\">Use insights to adjust the system and the process in real time<\/p>\n<\/li>\n<\/ul>\n<p data-start=\"4380\" data-end=\"4449\"><strong data-start=\"4380\" data-end=\"4449\">Evaluation is not the end of reform. It\u2019s how reform stays alive.<\/strong><\/p>\n<h4 data-start=\"4456\" data-end=\"4482\">Let\u2019s Make It Practical<\/h4>\n<p data-start=\"4484\" data-end=\"4551\">To test whether your digital reform is truly working, ask yourself:<\/p>\n<ul>\n<li data-start=\"4558\" data-end=\"4612\"><strong data-start=\"4564\" data-end=\"4610\">Are we measuring outcomes or just outputs?: <\/strong>It\u2019s not about counting transactions \u2014 it\u2019s about seeing if services are more accessible, transparent, and trusted.<\/li>\n<\/ul>\n<ul>\n<li data-start=\"4735\" data-end=\"4784\"><strong data-start=\"4741\" data-end=\"4782\">What behaviors have actually changed?: <\/strong>If staff still cling to manual processes or old workarounds, the tech isn\u2019t the problem \u2014 the adoption is.<\/li>\n<\/ul>\n<ul>\n<li data-start=\"4898\" data-end=\"4935\"><strong data-start=\"4904\" data-end=\"4933\">Who owns this reform now?: <\/strong>If the system depends on external contractors, what happens when the contract ends? Do internal teams have the skill and confidence to keep improving it?<\/li>\n<\/ul>\n<ul>\n<li data-start=\"5098\" data-end=\"5144\"><strong data-start=\"5104\" data-end=\"5142\">Is evaluation feeding improvement?: <\/strong>Are you using feedback and data to evolve the system, or are reports just being filed away?<\/li>\n<\/ul>\n<h4 data-start=\"5243\" data-end=\"5291\">Build Change, Not Just Systems<\/h4>\n<blockquote data-start=\"5293\" data-end=\"5417\">\n<h4 style=\"text-align: center;\" data-start=\"5295\" data-end=\"5417\"><em>\u201cA digital platform is not reform. Reform is when the institution itself learns, adapts, and serves better because of it.\u201d<\/em><\/h4>\n<\/blockquote>\n<p data-start=\"5419\" data-end=\"5696\">The goal of digital transformation isn\u2019t just to modernize technology \u2014 it\u2019s to modernize how we govern and how we deliver value to people. Measuring that requires us to look deeper, ask harder questions, and treat evaluation as part of leadership, not just project management.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Digital transformation is often celebrated as a sign of progress [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":169,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-37","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"rttpg_featured_image_url":{"full":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-scaled.jpg",2560,1707,false],"landscape":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-scaled.jpg",2560,1707,false],"portraits":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-scaled.jpg",2560,1707,false],"thumbnail":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-150x150.jpg",150,150,true],"medium":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-300x200.jpg",300,200,true],"large":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-1024x683.jpg",1024,683,true],"1536x1536":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-1536x1024.jpg",1536,1024,true],"2048x2048":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-2048x1366.jpg",2048,1366,true],"blog-large":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-669x272.jpg",669,272,true],"blog-medium":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-320x202.jpg",320,202,true],"recent-posts":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-700x441.jpg",700,441,true],"recent-works-thumbnail":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-66x66.jpg",66,66,true],"fusion-200":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-200x133.jpg",200,133,true],"fusion-400":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-400x267.jpg",400,267,true],"fusion-600":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-600x400.jpg",600,400,true],"fusion-800":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-800x534.jpg",800,534,true],"fusion-1200":["https:\/\/deguzman.ph\/wp-content\/uploads\/2023\/01\/measuring-different-reforms-1200x800.jpg",1200,800,true]},"rttpg_author":{"display_name":"deguzman","author_link":"https:\/\/deguzman.ph\/index.php\/author\/deguzman\/"},"rttpg_comment":2,"rttpg_category":"<a href=\"https:\/\/deguzman.ph\/index.php\/category\/uncategorized\/\" rel=\"category tag\">Uncategorized<\/a>","rttpg_excerpt":"Digital transformation is often celebrated as a sign of progress [&hellip;]","_links":{"self":[{"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/posts\/37","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/comments?post=37"}],"version-history":[{"count":5,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/posts\/37\/revisions"}],"predecessor-version":[{"id":170,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/posts\/37\/revisions\/170"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/media\/169"}],"wp:attachment":[{"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/media?parent=37"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/categories?post=37"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/deguzman.ph\/index.php\/wp-json\/wp\/v2\/tags?post=37"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}