Why does PyMatching perform much worse on CUDA-Q’s memory-circuit DEM than on Stim’s detector error model?

Why does PyMatching perform much worse on CUDA-Q’s memory-circuit DEM than on Stim’s detector error model? Ask Question Asked today Modified today Viewed 15 times 0 I am comparing logical error rates for a surface-code memory experiment using two pipelines: Stim + PyMatching CUDA-Q memory circuit DEM + PyMatching The code is import numpy as np import matplotlib.pyplot as plt import cudaq import cudaq_qec as qec import stim import pymatching # ============================================================ # Custom CUDA-Q decoder wrapping PyMatching and decoding # directly to observables # ============================================================ @qec.decoder("my_pymatching_obs") class MyPyMatchingObsDecoder: def __init__(self, H, **kwargs): qec.Decoder.__init__(self, H) self.H = np.array(H, dtype=np.uint8, order="C") if "O" not in kwargs: raise ValueError("my_pymatching_obs requires O=observables_flips_matrix") self.O = np.array(kwargs["O"], dtype=np.uint8, order="C") if self.O.ndim != 2: raise ValueError(f"O must be 2D, got shape {self.O.shape}") if self.O.shape[1] != self.H.shape[1]: raise ValueError( f"O must have same number of columns as H. " f"Got O.shape={self.O.shape}, H.shape={self.H.shape}" ) matching_kwargs = { "faults_matrix": self.O, } error_rate_vec = kwargs.get("error_rate_vec", None) if error_rate_vec is not None: p = np.array(error_rate_vec, dtype=float) if p.shape != (self.H.shape[1],): raise ValueError( f"error_rate_vec must have length {self.H.shape[1]}, got {p.shape}" ) eps = 1e-15 p = np.clip(p, eps, 1 - eps) weights = np.log((1.0 - p) / p) matching_kwargs["weights"] = weights matching_kwargs["error_probabilities"] = p self.matching = pymatching.Matching.from_check_matrix( self.H, **matching_kwargs, ) def decode(self, syndrome): syndrome = np.asarray(syndrome, dtype=np.uint8).reshape(-1) pred = self.matching.decode(syndrome) pred = np.asarray(pred, dtype=np.uint8).reshape(-1) result = qec.DecoderResult() result.converged = True result.result = pred.astype(float).tolist() return result # ============================================================ # Global settings # ============================================================ cudaq.set_target("stim") NUM_SHOTS = 1000 DISTANCES = [3, 5] PHYSICAL_ERROR_RATES = [0.002, 0.003, 0.005, 0.01] # ============================================================ # Stim + PyMatching # ============================================================ def count_logical_errors_stim(circuit: stim.Circuit, num_shots: int) -> int: sampler = circuit.compile_detector_sampler() detection_events, observable_flips = sampler.sample( num_shots, separate_observables=True ) detector_error_model = circuit.detector_error_model(decompose_errors=True) matcher = pymatching.Matching.from_detector_error_model(detector_error_model) predictions = matcher.decode_batch(detection_events) num_errors = 0 for shot in range(num_shots): actual_for_shot = observable_flips[shot] predicted_for_shot = predictions[shot] if not np.array_equal(actual_for_shot, predicted_for_shot): num_errors += 1 return num_errors def stim_rates(distance: int, p: float, num_shots: int): """ Returns: uncorrected_rate, corrected_rate """ circuit = stim.Circuit.generated( "surface_code:rotated_memory_z", rounds=distance * 3, distance=distance, after_clifford_depolarization=p, ) sampler = circuit.compile_detector_sampler() detection_events, observable_flips = sampler.sample( num_shots, separate_observables=True ) uncorrected_rate = np.sum(observable_flips) / num_shots detector_error_model = circuit.detector_error_model(decompose_errors=True) matcher = pymatching.Matching.from_detector_error_model(detector_error_model) predictions = matcher.decode_batch(detection_events) corrected_errors = 0 for shot in range(num_shots): actual_for_shot = observable_flips[shot] predicted_for_shot = predictions[shot] if not np.array_equal(actual_for_shot, predicted_for_shot): corrected_errors += 1 corrected_rate = corrected_errors / num_shots return uncorrected_rate, corrected_rate # ============================================================ # CUDA-Q + custom PyMatching decoder # ============================================================ def cudaq_rates(distance: int, p: float, num_shots: int): """ Returns: uncorrected_rate, corrected_rate """ code = qec.get_code("surface_code", distance=distance) Lz = code.get_observables_z() state_prep = qec.operation.prep0 n_rounds = 3 * distance noise = cudaq.NoiseModel() noise.add_all_qubit_channel("x", cudaq.Depolarization2(p), 1) dem = qec.z_dem_from_memory_circuit(code, state_prep, n_rounds, noise) syndromes, data = qec.sample_memory_circuit( code, state_prep, num_shots, n_rounds, noise, ) logical_measurements = (Lz @ data.transpose()) % 2 logical_measurements = logical_measurements.flatten().astype(np.uint8) uncorrected_rate = np.sum(logical_measurements) / num_shots # Keep only Z stabilizers for prep0 syndromes = syndromes.reshape((num_shots, n_rounds, -1)) syndromes = syndromes[:, :, :syndromes.shape[2] // 2] syndromes = syndromes.reshape((num_shots, -1)).astype(np.uint8) decoder = qec.get_decoder( "my_pymatching_obs", dem.detector_error_matrix, O=dem.observables_flips_matrix, error_rate_vec=np.array(dem.error_rates), ) dr = decoder.decode_batch(syndromes) obs_per_shot = np.array([r.result for r in dr], dtype=np.uint8) if obs_per_shot.ndim == 2 and obs_per_shot.shape[1] == 1: data_predictions = obs_per_shot[:, 0] else: data_predictions = obs_per_shot.flatten() corrected_rate = np.sum(data_predictions ^ logical_measurements) / num_shots return uncorrected_rate, corrected_rate # ============================================================ # Main # ============================================================ def main(): stim_uncorr = {} stim_corr = {} cudaq_uncorr = {} cudaq_corr = {} print("\n=== Stim + PyMatching ===") for d in DISTANCES: ys_u = [] ys_c = [] for p in PHYSICAL_ERROR_RATES: ru, rc = stim_rates(d, p, NUM_SHOTS) ys_u.append(ru) ys_c.append(rc) print( f"Stim+PM d={d}, p={p:.4f}, " f"uncorrected={ru:.6f}, corrected={rc:.6f}" ) stim_uncorr[d] = ys_u stim_corr[d] = ys_c print("\n=== CUDA-Q + custom PyMatching ===") for d in DISTANCES: ys_u = [] ys_c = [] for p in PHYSICAL_ERROR_RATES: ru, rc = cudaq_rates(d, p, NUM_SHOTS) ys_u.append(ru) ys_c.append(rc) print( f"CUDA-Q+PM d={d}, p={p:.4f}, " f"uncorrected={ru:.6f}, corrected={rc:.6f}" ) cudaq_uncorr[d] = ys_u cudaq_corr[d] = ys_c # -------------------------------------------------- # Figure 1: corrected logical error rate # -------------------------------------------------- plt.figure(figsize=(9, 6)) for d in DISTANCES: plt.plot( PHYSICAL_ERROR_RATES, stim_corr[d], marker="x", linestyle="--", label=f"Stim corrected d={d}", ) for d in DISTANCES: plt.plot( PHYSICAL_ERROR_RATES, cudaq_corr[d], marker="o", linestyle="-", label=f"CUDA-Q corrected d={d}", ) plt.loglog() plt.xlabel("physical error rate") plt.ylabel("logical error rate per shot") plt.title("Corrected logical error rate: Stim vs CUDA-Q") plt.grid(True, which="both", alpha=0.3) plt.legend() plt.tight_layout() # -------------------------------------------------- # Figure 2: uncorrected logical error rate # -------------------------------------------------- plt.figure(figsize=(9, 6)) for d in DISTANCES: plt.plot( PHYSICAL_ERROR_RATES, stim_uncorr[d], marker="x", linestyle="--", label=f"Stim uncorrected d={d}", ) for d in DISTANCES: plt.plot( PHYSICAL_ERROR_RATES, cudaq_uncorr[d], marker="o", linestyle="-", label=f"CUDA-Q uncorrected d={d}", ) plt.loglog() plt.xlabel("physical error rate") plt.ylabel("logical error rate per shot") plt.title("Uncorrected logical error rate: Stim vs CUDA-Q") plt.grid(True, which="both", alpha=0.3) plt.legend() plt.tight_layout() plt.show() if __name__ == "__main__": main() For comparison I show the logical errors with and without corrections. The uncorrected logical error rates are approximately equal. With correction, Stim perform better than Cuda-Q. Is this difference in performance expected when using PyMatching on a CUDA-Q DEM? error-correctionstimdecoding Share Improve this question Follow asked 2 hours ago MatthiasN 1 1 Your code reads like you're creating different circuits for the two different systems. It's not surprising you'd get different behavior. Rewrite the code so that both methods are given the same circuit, rather than each method making its own circuit. Also, your plots are missing error bars. You can't tell what's shot noise and what's signal without error bars. Add error bars. –  Craig Gidney Commented 49 mins ago Add a comment Know someone who can answer? Share a link to this question via email, Twitter, or Facebook. Your Answer Sign up or log in Sign up using Google Sign up using Email and Password Post as a guest Name Email Required, but never shown Post Your Answer By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy. Start asking to get answers Find the answer to your question by asking. Ask question Explore related questions error-correctionstimdecoding See similar questions with these tags. Related 1 Stim detector error model to syndrome graph 1 How to properly generate circuit measurement results from detector error model 5 What is the best way to parallelise processes across multiple cores when computing logical error rates with Stim and PyMatching? 4 Stim - Error Detector Model - Understanding approximate_disjoint_errors and decompose_errors 1 Detector error model for surface code with multiple rounds 0 Creating a Detector Error Model for a Surface Code Experiment 4 How is detector error model generated in Stim 1 Calculation of error probability in detector error model 4 What does red lines represent in detector error model diagram generated by stim? 2 Detector error model for a circuit with post selections Hot Network Questions firstNonNull implementations Was Fermat's Last Theorem known for infinitely many primes before Wiles? Alternative proof of the Riesz Representation Theorem What are these things on Chell's legs? Deriving helium atom wavefunction Replace spigot + galvanized supply line Why does gauge symmetries cause differential operator in action to be singular? Parent push bike where the pedals move WITH the wheels How to handle calendar year as a continuous predictor with a mismatched train/test time horizon? Adding "real" creation date to scanned "old"pictures Do those who later identify as “ex-Christians” remain saved under the “once saved, always saved” doctrine? Old story about space-walking humans Badgermole Cub with no mana from Gyre Sage Cryptic Family Reunion: Indulging in Tender Emotions What is the difference between a tautology and a definition? Why do larger language models still fail on simple compositional reasoning tasks? Are "very short stories" common in Arabic literature, or specifically an Alomar thing? mgt.clearMarks is not a function Short story from the 1930’s through mid 1950’s with a plot that includes robots learning how to replicate independently of humans? "Onomat-appear" - a straightforward wordsearch? Movement of space vehicles What role does transistor T2 play in this constant voltage reference generator? The Emperor’s Command: All Roads Lead to Rome Is it easy to change field of research when moving from PhD to postdoc? Question feed By continuing to use this website, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy. By exiting this window, default cookies will be accepted. To reject cookies, select an option from below. Customize settings Cookie Consent Preference Center When you visit any of our websites, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences, or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and manage your preferences. Please note, blocking some types of cookies may impact your experience of the site and the services we are able to offer. Cookie Policy Accept all cookies Manage Consent Preferences Strictly Necessary Cookies Always Active These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information. Targeting Cookies Targeting Cookies These cookies are used to make advertising messages more relevant to you and may be set through our site by us or by our advertising partners. They may be used to build a profile of your interests and show you relevant advertising on our site or on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. Performance Cookies Performance Cookies These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance. Functional Cookies Functional Cookies These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly. Cookie List Clear checkbox label label Apply Cancel Consent Leg.Interest checkbox label label checkbox label label checkbox label label Necessary cookies only Confirm My Choices