Jul 24 22:35:36.687506 ip-10-0-130-123 systemd[1]: Starting Kubernetes Kubelet... Jul 24 22:35:37.031451 ip-10-0-130-123 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:35:37.031451 ip-10-0-130-123 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 22:35:37.031451 ip-10-0-130-123 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:35:37.031451 ip-10-0-130-123 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 22:35:37.031451 ip-10-0-130-123 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:35:37.032326 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.032246 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 22:35:37.036085 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036070 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036087 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036091 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036093 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036096 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036099 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036102 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036104 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036107 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036109 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036112 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036114 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036117 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036119 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036123 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036125 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036127 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036130 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036133 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.036128 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036136 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036139 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036141 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036144 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036147 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036149 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036152 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036154 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036157 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036159 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036162 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036164 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036167 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036169 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036172 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036175 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036177 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036179 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036181 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036184 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.036592 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036187 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036189 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036191 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036194 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036196 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036198 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036201 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036203 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036205 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036208 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036210 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036212 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036215 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036218 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036234 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036237 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036240 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036243 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036245 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.037072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036248 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036252 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036256 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036259 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036262 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036264 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036267 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036269 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036272 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036274 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036276 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036279 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036284 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036288 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036291 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036294 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036297 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036300 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036302 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.037595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036305 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036307 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036309 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036312 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036314 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036317 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036319 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036321 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036326 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036329 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036332 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036334 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036337 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036340 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036344 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036347 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036349 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036352 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036355 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.038056 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036357 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036359 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.036362 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037058 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037064 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037067 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037070 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037072 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037076 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037078 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037082 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037085 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037087 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037090 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037092 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037095 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037097 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037100 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037103 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037105 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.038569 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037108 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037111 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037114 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037116 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037119 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037121 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037124 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037126 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037128 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037131 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037133 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037136 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037138 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037141 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037143 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037145 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037148 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037150 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037152 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037155 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.039047 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037157 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037161 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037163 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037165 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037168 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037170 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037172 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037175 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037177 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037180 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037182 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037186 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037190 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037194 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037197 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037200 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037203 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037205 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037208 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.039571 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037210 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037213 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037215 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037219 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037240 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037243 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037245 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037248 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037251 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037253 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037256 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037259 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037261 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037263 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037266 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037269 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037272 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037274 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037277 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.040048 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037281 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037283 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037286 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037288 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037291 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037294 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037296 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037299 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037301 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037304 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037306 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037310 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037313 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037316 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037319 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037321 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037324 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037326 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037329 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.040530 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037331 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037333 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037336 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037339 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037341 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037708 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037712 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037715 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037718 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037724 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037726 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037729 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037732 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037734 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037737 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037739 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037742 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037745 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037747 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.040993 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037750 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037754 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037758 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037761 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037763 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037766 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037768 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037771 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037773 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037775 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037778 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037781 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037784 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037788 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037791 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037794 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037797 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037799 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.041466 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037802 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037805 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037807 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037809 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037811 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037817 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037820 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037824 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037826 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037829 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037831 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037834 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037838 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037840 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037842 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037845 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037847 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037850 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037852 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037855 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.041899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037858 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037861 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037863 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037866 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037868 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037870 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037873 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037875 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037878 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037880 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037882 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037885 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037887 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037890 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037892 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037895 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037897 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037900 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037903 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037906 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.042388 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037909 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037911 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037913 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037916 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037920 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037923 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037925 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037928 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037930 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037933 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037935 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037938 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037940 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037942 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037945 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037947 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037953 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037955 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037958 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037960 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.042861 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037963 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037965 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037967 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037970 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037972 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037975 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.037977 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038049 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038057 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038062 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038067 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038072 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038075 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038079 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038083 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038086 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038090 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038093 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038096 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038099 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038102 2559 flags.go:64] FLAG: --cgroup-root="" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038105 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038107 2559 flags.go:64] FLAG: --client-ca-file="" Jul 24 22:35:37.043341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038110 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038113 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038117 2559 flags.go:64] FLAG: --cluster-domain="" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038120 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038123 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038126 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038129 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038133 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038135 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038138 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038141 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038144 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038147 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038149 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038152 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038155 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038159 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038162 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038164 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038167 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038170 2559 flags.go:64] FLAG: --enable-server="true" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038177 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038181 2559 flags.go:64] FLAG: --event-burst="100" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038184 2559 flags.go:64] FLAG: --event-qps="50" Jul 24 22:35:37.043876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038186 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038190 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038193 2559 flags.go:64] FLAG: --eviction-hard="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038197 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038199 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038202 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038205 2559 flags.go:64] FLAG: --eviction-soft="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038208 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038210 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038213 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038216 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038218 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038235 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038238 2559 flags.go:64] FLAG: --feature-gates="" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038241 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038244 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038247 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038250 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038253 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038255 2559 flags.go:64] FLAG: --help="false" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038258 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038272 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038276 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 22:35:37.044493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038279 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038282 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038285 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038288 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038291 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038294 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038297 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038315 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038318 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038321 2559 flags.go:64] FLAG: --kube-reserved="" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038325 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038328 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038331 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038333 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038336 2559 flags.go:64] FLAG: --lock-file="" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038339 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038342 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038345 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038354 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038357 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038360 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038363 2559 flags.go:64] FLAG: --logging-format="text" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038365 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038369 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 22:35:37.045101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038372 2559 flags.go:64] FLAG: --manifest-url="" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038374 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038378 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038381 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038385 2559 flags.go:64] FLAG: --max-pods="110" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038388 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038390 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038393 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038395 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038398 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038401 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038404 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038410 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038413 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038416 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038419 2559 flags.go:64] FLAG: --pod-cidr="" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038427 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038432 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038435 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038439 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038442 2559 flags.go:64] FLAG: --port="10250" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038445 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038448 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07720b9c9ac4209b1" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038451 2559 flags.go:64] FLAG: --qos-reserved="" Jul 24 22:35:37.045693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038453 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038456 2559 flags.go:64] FLAG: --register-node="true" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038459 2559 flags.go:64] FLAG: --register-with-taints="" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038462 2559 flags.go:64] FLAG: --registry-burst="10" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038465 2559 flags.go:64] FLAG: --registry-qps="5" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038468 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038470 2559 flags.go:64] FLAG: --reserved-memory="" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038474 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038476 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038479 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038482 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038484 2559 flags.go:64] FLAG: --runonce="false" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038487 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038490 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038495 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038497 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038500 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038503 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038506 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038509 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038511 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038514 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038516 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038519 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038522 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 22:35:37.046300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038530 2559 flags.go:64] FLAG: --system-cgroups="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038532 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038538 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038541 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038543 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038547 2559 flags.go:64] FLAG: --tls-min-version="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038550 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038553 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038556 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038558 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038561 2559 flags.go:64] FLAG: --v="2" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038564 2559 flags.go:64] FLAG: --version="false" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038568 2559 flags.go:64] FLAG: --vmodule="" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038572 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038575 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038707 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038711 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038714 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038717 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038719 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038722 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038726 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038729 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.046900 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038732 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038735 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038737 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038740 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038743 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038745 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038748 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038750 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038753 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038755 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038763 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038767 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038769 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038772 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038774 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038777 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038779 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038782 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038784 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038787 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038789 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.047513 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038792 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038794 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038797 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038799 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038802 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038805 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038807 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038810 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038812 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038816 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038818 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038821 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038823 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038825 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038827 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038830 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038833 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038835 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038837 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.048351 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038840 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038843 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038845 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038854 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038857 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038859 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038862 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038864 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038867 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038870 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038872 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038875 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038878 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038880 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038882 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038885 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038887 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038890 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038892 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038894 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.048899 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038897 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038899 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038903 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038905 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038908 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038912 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038915 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038918 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038922 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038925 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038927 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038930 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038932 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038934 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038937 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038941 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038944 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038947 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038949 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.049403 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038952 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038954 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038957 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038959 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038962 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038964 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038967 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038970 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038974 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038977 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038980 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.038982 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.038988 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.045650 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 22:35:37.049856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.045665 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.046360 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.046368 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046429 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046434 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046438 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046440 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046443 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046446 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046451 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046454 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046458 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046460 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046463 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046466 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046469 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046472 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046474 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046477 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.050199 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046479 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046482 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046484 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046487 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046489 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046492 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046494 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046497 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046500 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046502 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046504 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046507 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046509 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046513 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046515 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046518 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046521 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046523 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046526 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046529 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.050699 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046533 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046535 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046538 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046541 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046544 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046547 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046550 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046553 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046555 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046561 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046564 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046567 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046569 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046571 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046574 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046576 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046579 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046581 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046585 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.051196 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046588 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046591 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046594 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046596 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046599 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046601 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046604 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046607 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046610 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046612 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046615 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046617 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046620 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046623 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046625 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046627 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046630 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046632 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046635 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046638 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.051660 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046640 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046643 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046645 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046648 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046651 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046653 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046656 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046658 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046660 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046663 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046665 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046668 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046671 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046673 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046676 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046679 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046682 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046684 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046687 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046690 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.052174 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046692 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046695 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046697 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046700 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.046705 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046839 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046844 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046847 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046850 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046852 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046855 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046858 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046860 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046862 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046865 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:35:37.052774 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046868 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046876 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046879 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046881 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046883 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046886 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046888 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046892 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046895 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046898 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046902 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046904 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046907 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046910 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046912 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046915 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046918 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046921 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046923 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:35:37.053139 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046926 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046928 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046930 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046933 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046935 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046937 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046940 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046942 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046945 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046947 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046950 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046952 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046955 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046957 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046959 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046968 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046970 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046973 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046975 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:35:37.053619 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046977 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046980 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046982 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046985 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046987 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046989 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046992 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046995 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.046997 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047000 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047003 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047005 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047007 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047010 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047013 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047015 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047017 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047020 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047022 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047024 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:35:37.054077 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047027 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047029 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047031 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047034 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047036 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047038 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047041 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047043 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047045 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047055 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047058 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047061 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047063 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047067 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047070 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047072 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047075 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047078 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047081 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047084 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:35:37.054676 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047086 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047089 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047092 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047094 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047097 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047100 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047102 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047105 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047107 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047111 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:37.047114 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.047119 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.047221 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.050279 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 22:35:37.055168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.051628 2559 server.go:1019] "Starting client certificate rotation" Jul 24 22:35:37.055542 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.051730 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 22:35:37.055542 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.051763 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 22:35:37.055542 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.054008 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 22:35:37.055542 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.055123 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 22:35:37.063323 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.063301 2559 log.go:25] "Validated CRI v1 runtime API" Jul 24 22:35:37.065739 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.065723 2559 log.go:25] "Validated CRI v1 image API" Jul 24 22:35:37.066578 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.066563 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 22:35:37.070360 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.070335 2559 fs.go:135] Filesystem UUIDs: map[07a1758d-3f9a-4f70-b787-06c1bdc30c37:/dev/nvme0n1p4 33d72355-5fb4-440f-8481-3ad31a0ef00a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 24 22:35:37.070360 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.070353 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 22:35:37.077035 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.076924 2559 manager.go:216] Machine: {Timestamp:2026-07-24 22:35:37.074486684 +0000 UTC m=+0.281725483 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098200 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2778176fd271d6768d876e305c8772 SystemUUID:ec277817-6fd2-71d6-768d-876e305c8772 BootID:969f0a09-8cd4-47de-aaa5-1058898d73bb Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f2:7e:c8:4c:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f2:7e:c8:4c:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:8b:88:12:c3:fc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 22:35:37.077035 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.077032 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 22:35:37.077138 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.077105 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 22:35:37.079763 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.079739 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 22:35:37.079905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.079769 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-123.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 22:35:37.080370 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.080359 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 22:35:37.080413 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.080373 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 22:35:37.080413 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.080381 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 22:35:37.080932 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.080920 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 22:35:37.082002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.081992 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 22:35:37.082791 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.082775 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 22:35:37.084749 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084737 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 24 22:35:37.084883 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084873 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 22:35:37.084917 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084911 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 24 22:35:37.084954 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084934 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 22:35:37.084985 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084953 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 22:35:37.084985 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084972 2559 kubelet.go:398] "Adding apiserver pod source" Jul 24 22:35:37.085037 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.084991 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 22:35:37.085444 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.085429 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 22:35:37.085524 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.085448 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 22:35:37.088268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.088120 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 22:35:37.090853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.090841 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 22:35:37.090945 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.090869 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 22:35:37.092438 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092425 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092441 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092448 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092453 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092461 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092466 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092473 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092479 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092489 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092496 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092507 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 22:35:37.092507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.092512 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 22:35:37.093204 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.093192 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 22:35:37.093256 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.093206 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 22:35:37.096373 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.096360 2559 server.go:1268] "Started kubelet" Jul 24 22:35:37.096446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.096424 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 22:35:37.096552 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.096498 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 22:35:37.096647 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.096566 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 22:35:37.097565 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.097547 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-123.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 22:35:37.097650 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.097590 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 22:35:37.097650 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.097590 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 22:35:37.097783 ip-10-0-130-123 systemd[1]: Started Kubernetes Kubelet. Jul 24 22:35:37.098397 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.098301 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 22:35:37.099343 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.099326 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 24 22:35:37.101344 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.100445 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-123.ec2.internal.18c55aaf7da9efa3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-123.ec2.internal,UID:ip-10-0-130-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-123.ec2.internal,},FirstTimestamp:2026-07-24 22:35:37.096335267 +0000 UTC m=+0.303574066,LastTimestamp:2026-07-24 22:35:37.096335267 +0000 UTC m=+0.303574066,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-123.ec2.internal,}" Jul 24 22:35:37.101733 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.101714 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 22:35:37.105891 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.105874 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 22:35:37.105974 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.105892 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 22:35:37.106292 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.106270 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.106672 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106652 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 22:35:37.106772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106673 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 22:35:37.106772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106690 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 22:35:37.106772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106731 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 22:35:37.106772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106741 2559 factory.go:55] Registering systemd factory Jul 24 22:35:37.106772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106751 2559 factory.go:223] Registration of the systemd container factory successfully Jul 24 22:35:37.106999 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106803 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 24 22:35:37.106999 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.106814 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 24 22:35:37.107125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.107012 2559 factory.go:153] Registering CRI-O factory Jul 24 22:35:37.107125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.107031 2559 factory.go:223] Registration of the crio container factory successfully Jul 24 22:35:37.107125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.107068 2559 factory.go:103] Registering Raw factory Jul 24 22:35:37.107125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.107083 2559 manager.go:1196] Started watching for new ooms in manager Jul 24 22:35:37.107696 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.107673 2559 manager.go:349] Starting recovery of all containers Jul 24 22:35:37.107847 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.107826 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-123.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 22:35:37.107996 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.107978 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 22:35:37.117451 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.117436 2559 manager.go:354] Recovery completed Jul 24 22:35:37.125069 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.124966 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.126831 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.126815 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.126891 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.126845 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.126891 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.126855 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.127502 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.127483 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 22:35:37.127502 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.127498 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.127633 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.127502 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 22:35:37.127633 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.127564 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 24 22:35:37.128164 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.128150 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.128264 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.128176 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.128264 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.128186 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.130315 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.130303 2559 policy_none.go:49] "None policy: Start" Jul 24 22:35:37.130353 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.130318 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 22:35:37.130353 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.130328 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 22:35:37.132281 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.132271 2559 policy_none.go:47] "Start" Jul 24 22:35:37.137308 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.137291 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9qdf6" Jul 24 22:35:37.139153 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.139086 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-123.ec2.internal.18c55aaf7f7b49c9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-123.ec2.internal,UID:ip-10-0-130-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-123.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-123.ec2.internal,},FirstTimestamp:2026-07-24 22:35:37.126832585 +0000 UTC m=+0.334071385,LastTimestamp:2026-07-24 22:35:37.126832585 +0000 UTC m=+0.334071385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-123.ec2.internal,}" Jul 24 22:35:37.146830 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.146811 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9qdf6" Jul 24 22:35:37.151788 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.151724 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-123.ec2.internal.18c55aaf7f7b8d22 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-123.ec2.internal,UID:ip-10-0-130-123.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-123.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-123.ec2.internal,},FirstTimestamp:2026-07-24 22:35:37.126849826 +0000 UTC m=+0.334088625,LastTimestamp:2026-07-24 22:35:37.126849826 +0000 UTC m=+0.334088625,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-123.ec2.internal,}" Jul 24 22:35:37.166745 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.166731 2559 manager.go:337] "Starting Device Plugin manager" Jul 24 22:35:37.166808 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.166757 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 22:35:37.166808 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.166767 2559 server.go:87] "Starting device plugin registration server" Jul 24 22:35:37.166996 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.166986 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 22:35:37.167032 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.166999 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 22:35:37.167161 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.167144 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 22:35:37.167268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.167243 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 22:35:37.167268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.167254 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 22:35:37.167723 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.167705 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 22:35:37.167808 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.167755 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.205930 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.205904 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 22:35:37.213205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.207139 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 22:35:37.213205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.207169 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 22:35:37.213205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.207185 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 22:35:37.213205 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.207215 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 22:35:37.213205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.209598 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:35:37.267585 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.267552 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.268363 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.268350 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.268422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.268377 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.268422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.268387 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.268422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.268409 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.278076 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.278062 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.278120 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.278087 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-123.ec2.internal\": node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.307621 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.307568 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal"] Jul 24 22:35:37.307703 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.307625 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.308360 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.308343 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.308422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.308372 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.308422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.308381 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.310628 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.310616 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.310738 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.310723 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.310774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.310754 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.311277 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311262 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.311366 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311292 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.311366 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311307 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.311366 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311266 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.311366 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311357 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.311366 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.311366 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.313488 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.313473 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.313564 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.313505 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:35:37.314184 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.314165 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:35:37.314290 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.314196 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:35:37.314290 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.314211 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:35:37.316774 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.316757 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.341798 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.341780 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-123.ec2.internal\" not found" node="ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.346171 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.346157 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-123.ec2.internal\" not found" node="ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.408053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.408038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.408125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.408060 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.408125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.408076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542ff756433833167701e75caa9f94dd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-123.ec2.internal\" (UID: \"542ff756433833167701e75caa9f94dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.416814 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.416795 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.509211 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509190 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.509284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542ff756433833167701e75caa9f94dd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-123.ec2.internal\" (UID: \"542ff756433833167701e75caa9f94dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.509284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.509343 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509290 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.509343 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509292 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e7e8d9d8bde8f18f73bb1ee2be3e58e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal\" (UID: \"6e7e8d9d8bde8f18f73bb1ee2be3e58e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.509343 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.509326 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542ff756433833167701e75caa9f94dd-config\") pod \"kube-apiserver-proxy-ip-10-0-130-123.ec2.internal\" (UID: \"542ff756433833167701e75caa9f94dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.517297 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.517282 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.618052 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.618003 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.645191 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.645175 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.650336 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.650110 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:37.718688 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.718665 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.819142 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.819116 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.919698 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:37.919643 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-123.ec2.internal\" not found" Jul 24 22:35:37.974121 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:37.974099 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:35:38.006213 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.006193 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" Jul 24 22:35:38.016584 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.016567 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 22:35:38.018200 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.018171 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" Jul 24 22:35:38.035851 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.035834 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 22:35:38.050989 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.050972 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 22:35:38.051099 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.051080 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 22:35:38.085931 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.085914 2559 apiserver.go:52] "Watching apiserver" Jul 24 22:35:38.096070 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.096050 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 22:35:38.098133 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.096805 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal","openshift-multus/multus-additional-cni-plugins-jhbt8","openshift-multus/multus-cgnnz","kube-system/konnectivity-agent-shbtf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht","openshift-multus/network-metrics-daemon-bvd5v","openshift-network-diagnostics/network-check-target-vpb69","openshift-network-operator/iptables-alerter-m6hxl","openshift-ovn-kubernetes/ovnkube-node-lplhc","kube-system/global-pull-secret-syncer-mfq5d","kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal","openshift-cluster-node-tuning-operator/tuned-wvsh9","openshift-dns/node-resolver-kv4dv","openshift-image-registry/node-ca-v5wqk"] Jul 24 22:35:38.099602 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.099581 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.101740 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.101722 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.103755 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.103737 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:38.103835 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.103791 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:38.105828 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.105810 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.105828 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.105826 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 22:35:38.105958 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.105899 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wk9ch\"" Jul 24 22:35:38.106008 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.105976 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 22:35:38.107122 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.107098 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 22:35:38.107214 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.107099 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 22:35:38.107301 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.107173 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 22:35:38.107301 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.107240 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.108643 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.108625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.110875 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.110859 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.110981 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.110917 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:38.110981 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.110967 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qxhhx\"" Jul 24 22:35:38.111782 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.111761 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.112069 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112054 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j2v87\"" Jul 24 22:35:38.112126 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112113 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112407 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112418 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112424 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112455 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112494 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-daemon-config\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112526 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-etc-kubernetes\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112551 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nljgm\" (UniqueName: \"kubernetes.io/projected/5e4b8fc2-3980-4526-8d47-c519c7920505-kube-api-access-nljgm\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.112606 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-cnibin\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112617 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-system-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112682 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw8xk\" (UniqueName: \"kubernetes.io/projected/847ae444-8ecf-4e2d-85a2-7e42057e472e-kube-api-access-zw8xk\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112712 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-multus\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-os-release\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112713 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nvnv8\"" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112793 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112824 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-device-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-netns\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112881 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112907 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-registration-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112931 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-cni-binary-copy\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.112970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-multus-certs\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-agent-certs\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113040 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-konnectivity-ca\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-system-cni-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113106 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-kubelet\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113140 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-os-release\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-socket-dir-parent\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113204 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-k8s-cni-cncf-io\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-conf-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113280 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113280 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.113548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113511 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-cnibin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-socket-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-hostroot\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113646 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113684 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-bin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113735 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87vq\" (UniqueName: \"kubernetes.io/projected/acb8d463-3918-4645-afd2-26855ea8f5dc-kube-api-access-p87vq\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.114006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.113766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-sys-fs\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.115609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.115584 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.117339 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.117323 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.117556 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.117536 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 22:35:38.117721 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.117706 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.117773 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.117758 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:38.118197 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.118184 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fthbz\"" Jul 24 22:35:38.118709 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.118695 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.118963 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.118947 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.119012 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.119004 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 22:35:38.119064 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.119013 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kpn8l\"" Jul 24 22:35:38.120058 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.120037 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 22:35:38.120058 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.120051 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 22:35:38.120183 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.120047 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.120609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.120594 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.121131 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.121114 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 22:35:38.123370 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.123354 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.125587 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.125567 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.128057 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.127706 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5nz88\"" Jul 24 22:35:38.128057 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.127715 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.128057 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.127750 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pqsl2\"" Jul 24 22:35:38.128057 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.127823 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.128057 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.127961 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.128272 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.128126 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.133869 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.133854 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 22:35:38.133943 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.133874 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 22:35:38.135246 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.135214 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 22:35:38.135246 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.135241 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l7tc2\"" Jul 24 22:35:38.135246 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.135246 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 22:35:38.152353 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.152333 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 22:30:37 +0000 UTC" deadline="2028-03-04 09:45:11.042194655 +0000 UTC" Jul 24 22:35:38.152353 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.152352 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14123h9m32.889845383s" Jul 24 22:35:38.204306 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.204282 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcwg5" Jul 24 22:35:38.208003 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.207985 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 22:35:38.213959 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.213933 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87vq\" (UniqueName: \"kubernetes.io/projected/acb8d463-3918-4645-afd2-26855ea8f5dc-kube-api-access-p87vq\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.214056 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.213978 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-kubelet-config\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.214056 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214009 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214056 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbsvj\" (UniqueName: \"kubernetes.io/projected/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-kube-api-access-fbsvj\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.214209 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nljgm\" (UniqueName: \"kubernetes.io/projected/5e4b8fc2-3980-4526-8d47-c519c7920505-kube-api-access-nljgm\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214209 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-cnibin\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.214209 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214108 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-system-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214209 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214178 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-system-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214198 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-cnibin\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214248 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw8xk\" (UniqueName: \"kubernetes.io/projected/847ae444-8ecf-4e2d-85a2-7e42057e472e-kube-api-access-zw8xk\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214292 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-bin\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysconfig\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214340 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-os-release\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214361 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-netns\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214384 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-systemd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-registration-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214433 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-config\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214468 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-os-release\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214479 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-modprobe-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214503 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-kubernetes\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-cni-binary-copy\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214547 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-registration-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214595 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-multus-certs\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214642 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-konnectivity-ca\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214646 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-multus-certs\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214689 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjfl\" (UniqueName: \"kubernetes.io/projected/65d397ae-e13d-4b6a-8147-ada839348d63-kube-api-access-7sjfl\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214703 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-netd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-script-lib\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.214781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-tmp\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214814 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-system-cni-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214857 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-socket-dir-parent\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-k8s-cni-cncf-io\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214906 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-conf-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214920 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-system-cni-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214948 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-socket-dir-parent\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-socket-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214969 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-conf-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/65d397ae-e13d-4b6a-8147-ada839348d63-iptables-alerter-script\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.214991 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-k8s-cni-cncf-io\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-conf\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215051 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-socket-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215052 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-kubelet\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215089 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-systemd-units\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215105 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-cni-binary-copy\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.215463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215121 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-var-lib-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215126 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-konnectivity-ca\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-hostroot\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215168 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215182 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpjjs\" (UniqueName: \"kubernetes.io/projected/6b2c4ffd-d9f2-459b-b1b2-290086b28121-kube-api-access-mpjjs\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-hosts-file\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215255 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215321 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-hostroot\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-sys-fs\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54tw6\" (UniqueName: \"kubernetes.io/projected/b0936fc0-4e09-4ec8-9759-183233c4020f-kube-api-access-54tw6\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215454 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215473 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-tmp-dir\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-daemon-config\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215535 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-sys-fs\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215563 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-etc-kubernetes\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215671 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-etc-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.216269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-lib-modules\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215737 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-multus\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215785 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-device-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215827 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-multus\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215841 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm8zt\" (UniqueName: \"kubernetes.io/projected/ad0f0815-c320-4476-8580-cc6f0504457f-kube-api-access-gm8zt\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215867 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-etc-kubernetes\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215869 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovn-node-metrics-cert\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-systemd\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215914 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-device-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-var-lib-kubelet\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215935 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-daemon-config\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215960 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-host\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.215997 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-netns\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216066 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-run-netns\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.216975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-log-socket\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216118 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-sys\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-agent-certs\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216171 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0936fc0-4e09-4ec8-9759-183233c4020f-serviceca\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/65d397ae-e13d-4b6a-8147-ada839348d63-host-slash\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216216 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-node-log\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-kubelet\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-os-release\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216311 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-dbus\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216365 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-env-overrides\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-cnibin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216497 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0936fc0-4e09-4ec8-9759-183233c4020f-host\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216503 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-slash\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.217840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqvp4\" (UniqueName: \"kubernetes.io/projected/4f947503-67bd-4d12-88c7-9bf939ed72b6-kube-api-access-mqvp4\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216564 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-run\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-tuned\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216602 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-ovn\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216681 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-bin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216737 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216903 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/acb8d463-3918-4645-afd2-26855ea8f5dc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216938 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-cni-binary-copy\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-kubelet\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.216972 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-os-release\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.217009 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/847ae444-8ecf-4e2d-85a2-7e42057e472e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.217025 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-cnibin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.217026 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-host-var-lib-cni-bin\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.217134 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e4b8fc2-3980-4526-8d47-c519c7920505-multus-cni-dir\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.218507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.217386 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/acb8d463-3918-4645-afd2-26855ea8f5dc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.219148 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.219024 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcwg5" Jul 24 22:35:38.219911 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.219895 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8cde2a49-cf6d-440a-9b1b-98f9539f7ef4-agent-certs\") pod \"konnectivity-agent-shbtf\" (UID: \"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4\") " pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.230784 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.230767 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:38.230866 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.230788 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:38.230866 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.230800 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:38.230981 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.230883 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:35:38.730856663 +0000 UTC m=+1.938095469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:38.234058 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.234029 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw8xk\" (UniqueName: \"kubernetes.io/projected/847ae444-8ecf-4e2d-85a2-7e42057e472e-kube-api-access-zw8xk\") pod \"aws-ebs-csi-driver-node-v8qht\" (UID: \"847ae444-8ecf-4e2d-85a2-7e42057e472e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.234168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.234151 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87vq\" (UniqueName: \"kubernetes.io/projected/acb8d463-3918-4645-afd2-26855ea8f5dc-kube-api-access-p87vq\") pod \"multus-additional-cni-plugins-jhbt8\" (UID: \"acb8d463-3918-4645-afd2-26855ea8f5dc\") " pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.238754 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.238735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nljgm\" (UniqueName: \"kubernetes.io/projected/5e4b8fc2-3980-4526-8d47-c519c7920505-kube-api-access-nljgm\") pod \"multus-cgnnz\" (UID: \"5e4b8fc2-3980-4526-8d47-c519c7920505\") " pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.296038 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.296021 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:35:38.301051 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.301021 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542ff756433833167701e75caa9f94dd.slice/crio-ce7dc1a8a9d4ac49162d8fd435d83ab73fed87bd221cb3a27052bbd945ddeddd WatchSource:0}: Error finding container ce7dc1a8a9d4ac49162d8fd435d83ab73fed87bd221cb3a27052bbd945ddeddd: Status 404 returned error can't find the container with id ce7dc1a8a9d4ac49162d8fd435d83ab73fed87bd221cb3a27052bbd945ddeddd Jul 24 22:35:38.301398 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.301381 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7e8d9d8bde8f18f73bb1ee2be3e58e.slice/crio-6cd1065132e5f26fd7b239bc30c619ea137f4e54235ba2fa3df5426f60a7d91d WatchSource:0}: Error finding container 6cd1065132e5f26fd7b239bc30c619ea137f4e54235ba2fa3df5426f60a7d91d: Status 404 returned error can't find the container with id 6cd1065132e5f26fd7b239bc30c619ea137f4e54235ba2fa3df5426f60a7d91d Jul 24 22:35:38.306585 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.306569 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 22:35:38.317791 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-dbus\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.317877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.317877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-env-overrides\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.317877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317839 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0936fc0-4e09-4ec8-9759-183233c4020f-host\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.317877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317860 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-slash\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317881 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqvp4\" (UniqueName: \"kubernetes.io/projected/4f947503-67bd-4d12-88c7-9bf939ed72b6-kube-api-access-mqvp4\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317903 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-run\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317908 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-dbus\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317913 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0936fc0-4e09-4ec8-9759-183233c4020f-host\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317901 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317927 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-slash\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.317956 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-run\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318042 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-tuned\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-ovn\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318085 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318104 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-ovn\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-kubelet-config\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318165 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbsvj\" (UniqueName: \"kubernetes.io/projected/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-kube-api-access-fbsvj\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-bin\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318202 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysconfig\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318218 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318238 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d839f85-78a3-4588-b9d7-b7de549c7d3d-kubelet-config\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318268 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318272 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-bin\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318299 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-netns\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-systemd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318356 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318359 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-run-netns\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318374 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-run-systemd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318329 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysconfig\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.318525 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318406 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-env-overrides\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318398 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318444 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-config\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318466 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-modprobe-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-kubernetes\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318515 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjfl\" (UniqueName: \"kubernetes.io/projected/65d397ae-e13d-4b6a-8147-ada839348d63-kube-api-access-7sjfl\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-netd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318557 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-kubernetes\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318563 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-script-lib\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-tmp\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.318603 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.318654 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:35:38.818641092 +0000 UTC m=+2.025879878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318685 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-cni-netd\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318741 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-modprobe-d\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318785 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/65d397ae-e13d-4b6a-8147-ada839348d63-iptables-alerter-script\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-conf\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319187 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-kubelet\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-systemd-units\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-var-lib-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpjjs\" (UniqueName: \"kubernetes.io/projected/6b2c4ffd-d9f2-459b-b1b2-290086b28121-kube-api-access-mpjjs\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318909 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-hosts-file\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54tw6\" (UniqueName: \"kubernetes.io/projected/b0936fc0-4e09-4ec8-9759-183233c4020f-kube-api-access-54tw6\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.318984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-tmp-dir\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319010 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-etc-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-lib-modules\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm8zt\" (UniqueName: \"kubernetes.io/projected/ad0f0815-c320-4476-8580-cc6f0504457f-kube-api-access-gm8zt\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319069 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-host-kubelet\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319087 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovn-node-metrics-cert\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-systemd\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-var-lib-kubelet\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319158 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-host\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319206 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-var-lib-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319245 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-host\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.319972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319247 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-hosts-file\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319294 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-etc-openvswitch\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319361 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/65d397ae-e13d-4b6a-8147-ada839348d63-iptables-alerter-script\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319393 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-lib-modules\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319034 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-sysctl-conf\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319428 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-tmp-dir\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319399 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-log-socket\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319482 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-sys\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319509 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0936fc0-4e09-4ec8-9759-183233c4020f-serviceca\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319527 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-log-socket\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/65d397ae-e13d-4b6a-8147-ada839348d63-host-slash\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319114 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-systemd-units\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.319552 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319561 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-node-log\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319588 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-systemd\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.319599 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:35:38.819583478 +0000 UTC m=+2.026822278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319637 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/65d397ae-e13d-4b6a-8147-ada839348d63-host-slash\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319638 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-sys\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.320601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319647 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4f947503-67bd-4d12-88c7-9bf939ed72b6-node-log\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319698 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b2c4ffd-d9f2-459b-b1b2-290086b28121-var-lib-kubelet\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.319942 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-config\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.320131 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0936fc0-4e09-4ec8-9759-183233c4020f-serviceca\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.320292 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovnkube-script-lib\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.320801 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-etc-tuned\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.321437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.321005 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2c4ffd-d9f2-459b-b1b2-290086b28121-tmp\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.321891 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.321876 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f947503-67bd-4d12-88c7-9bf939ed72b6-ovn-node-metrics-cert\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.330587 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.330566 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqvp4\" (UniqueName: \"kubernetes.io/projected/4f947503-67bd-4d12-88c7-9bf939ed72b6-kube-api-access-mqvp4\") pod \"ovnkube-node-lplhc\" (UID: \"4f947503-67bd-4d12-88c7-9bf939ed72b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.332121 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.332097 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbsvj\" (UniqueName: \"kubernetes.io/projected/5f4f9dfe-beef-4b50-a0e3-62f8d61e0178-kube-api-access-fbsvj\") pod \"node-resolver-kv4dv\" (UID: \"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178\") " pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.332285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.332271 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjfl\" (UniqueName: \"kubernetes.io/projected/65d397ae-e13d-4b6a-8147-ada839348d63-kube-api-access-7sjfl\") pod \"iptables-alerter-m6hxl\" (UID: \"65d397ae-e13d-4b6a-8147-ada839348d63\") " pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.336726 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.336709 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpjjs\" (UniqueName: \"kubernetes.io/projected/6b2c4ffd-d9f2-459b-b1b2-290086b28121-kube-api-access-mpjjs\") pod \"tuned-wvsh9\" (UID: \"6b2c4ffd-d9f2-459b-b1b2-290086b28121\") " pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.338396 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.338380 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm8zt\" (UniqueName: \"kubernetes.io/projected/ad0f0815-c320-4476-8580-cc6f0504457f-kube-api-access-gm8zt\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.338957 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.338938 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54tw6\" (UniqueName: \"kubernetes.io/projected/b0936fc0-4e09-4ec8-9759-183233c4020f-kube-api-access-54tw6\") pod \"node-ca-v5wqk\" (UID: \"b0936fc0-4e09-4ec8-9759-183233c4020f\") " pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.393016 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.392991 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:35:38.427401 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.427375 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cgnnz" Jul 24 22:35:38.437541 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.437513 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4b8fc2_3980_4526_8d47_c519c7920505.slice/crio-7a410c096f0d0a7a4a7ba5c74eeabf507e7abaf09ddd025fece71881b1a5a1c5 WatchSource:0}: Error finding container 7a410c096f0d0a7a4a7ba5c74eeabf507e7abaf09ddd025fece71881b1a5a1c5: Status 404 returned error can't find the container with id 7a410c096f0d0a7a4a7ba5c74eeabf507e7abaf09ddd025fece71881b1a5a1c5 Jul 24 22:35:38.447967 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.447950 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" Jul 24 22:35:38.457490 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.457469 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb8d463_3918_4645_afd2_26855ea8f5dc.slice/crio-dbe753ff952a8ef8b0b055099e9a68373f95eb2e6c8cd9270a9c2d9be3a284ff WatchSource:0}: Error finding container dbe753ff952a8ef8b0b055099e9a68373f95eb2e6c8cd9270a9c2d9be3a284ff: Status 404 returned error can't find the container with id dbe753ff952a8ef8b0b055099e9a68373f95eb2e6c8cd9270a9c2d9be3a284ff Jul 24 22:35:38.469666 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.469647 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:35:38.478152 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.478130 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cde2a49_cf6d_440a_9b1b_98f9539f7ef4.slice/crio-17c1bed6416f3d05ba1df2ba1e53ebd9024a09f4f85e486b9c925d7e4ef6c7fa WatchSource:0}: Error finding container 17c1bed6416f3d05ba1df2ba1e53ebd9024a09f4f85e486b9c925d7e4ef6c7fa: Status 404 returned error can't find the container with id 17c1bed6416f3d05ba1df2ba1e53ebd9024a09f4f85e486b9c925d7e4ef6c7fa Jul 24 22:35:38.481324 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.481302 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" Jul 24 22:35:38.486709 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.486685 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6hxl" Jul 24 22:35:38.491846 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.491827 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847ae444_8ecf_4e2d_85a2_7e42057e472e.slice/crio-eeb832e9b95fdf769cbeeb1ff9293681e3b36b57f53bdf109497435a3b6989fa WatchSource:0}: Error finding container eeb832e9b95fdf769cbeeb1ff9293681e3b36b57f53bdf109497435a3b6989fa: Status 404 returned error can't find the container with id eeb832e9b95fdf769cbeeb1ff9293681e3b36b57f53bdf109497435a3b6989fa Jul 24 22:35:38.496110 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.496089 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d397ae_e13d_4b6a_8147_ada839348d63.slice/crio-a960dfaa41633016485595d72e7760355f6f00b8627a56dd61617772f44a30df WatchSource:0}: Error finding container a960dfaa41633016485595d72e7760355f6f00b8627a56dd61617772f44a30df: Status 404 returned error can't find the container with id a960dfaa41633016485595d72e7760355f6f00b8627a56dd61617772f44a30df Jul 24 22:35:38.509844 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.509828 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:35:38.518743 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.518723 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f947503_67bd_4d12_88c7_9bf939ed72b6.slice/crio-c9fd070f32c780b07eacfbca8e703a346fba596259c06cacd28642b949d5aff7 WatchSource:0}: Error finding container c9fd070f32c780b07eacfbca8e703a346fba596259c06cacd28642b949d5aff7: Status 404 returned error can't find the container with id c9fd070f32c780b07eacfbca8e703a346fba596259c06cacd28642b949d5aff7 Jul 24 22:35:38.521019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.521002 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" Jul 24 22:35:38.528890 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.528871 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2c4ffd_d9f2_459b_b1b2_290086b28121.slice/crio-d6c64fb9e4381dd3877eeb85f37f0bee2f7edea9fe48320bf658d87ffb752053 WatchSource:0}: Error finding container d6c64fb9e4381dd3877eeb85f37f0bee2f7edea9fe48320bf658d87ffb752053: Status 404 returned error can't find the container with id d6c64fb9e4381dd3877eeb85f37f0bee2f7edea9fe48320bf658d87ffb752053 Jul 24 22:35:38.545555 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.545533 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kv4dv" Jul 24 22:35:38.548159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.548144 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5wqk" Jul 24 22:35:38.555065 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.555046 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4f9dfe_beef_4b50_a0e3_62f8d61e0178.slice/crio-619d057a71ba77010a3b4729e640ba8da994a8bedd42a06a6f899647f2f8d365 WatchSource:0}: Error finding container 619d057a71ba77010a3b4729e640ba8da994a8bedd42a06a6f899647f2f8d365: Status 404 returned error can't find the container with id 619d057a71ba77010a3b4729e640ba8da994a8bedd42a06a6f899647f2f8d365 Jul 24 22:35:38.557024 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:35:38.557004 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0936fc0_4e09_4ec8_9759_183233c4020f.slice/crio-4766db73cbbe8253c4a27d9acaa8228bd88ebbe63ce9af182ff6ef08ee04b078 WatchSource:0}: Error finding container 4766db73cbbe8253c4a27d9acaa8228bd88ebbe63ce9af182ff6ef08ee04b078: Status 404 returned error can't find the container with id 4766db73cbbe8253c4a27d9acaa8228bd88ebbe63ce9af182ff6ef08ee04b078 Jul 24 22:35:38.822987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.822889 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:38.822987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.822945 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:38.823204 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:38.823017 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:38.823204 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823150 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:38.823354 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823214 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:35:39.823195209 +0000 UTC m=+3.030433998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:38.823719 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823576 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:38.823719 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823673 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:35:39.823654098 +0000 UTC m=+3.030892899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:38.823719 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823691 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:38.823719 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823714 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:38.824075 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823728 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:38.824075 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:38.823782 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:35:39.82376665 +0000 UTC m=+3.031005453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:39.059554 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.059520 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:35:39.216567 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.216462 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cgnnz" event={"ID":"5e4b8fc2-3980-4526-8d47-c519c7920505","Type":"ContainerStarted","Data":"7a410c096f0d0a7a4a7ba5c74eeabf507e7abaf09ddd025fece71881b1a5a1c5"} Jul 24 22:35:39.224325 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.224271 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:30:38 +0000 UTC" deadline="2028-05-03 01:29:04.446687252 +0000 UTC" Jul 24 22:35:39.224325 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.224305 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15554h53m25.222386843s" Jul 24 22:35:39.234707 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.234675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" event={"ID":"542ff756433833167701e75caa9f94dd","Type":"ContainerStarted","Data":"ce7dc1a8a9d4ac49162d8fd435d83ab73fed87bd221cb3a27052bbd945ddeddd"} Jul 24 22:35:39.243711 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.243678 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" event={"ID":"6e7e8d9d8bde8f18f73bb1ee2be3e58e","Type":"ContainerStarted","Data":"6cd1065132e5f26fd7b239bc30c619ea137f4e54235ba2fa3df5426f60a7d91d"} Jul 24 22:35:39.246501 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.246477 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"c9fd070f32c780b07eacfbca8e703a346fba596259c06cacd28642b949d5aff7"} Jul 24 22:35:39.261166 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.261137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6hxl" event={"ID":"65d397ae-e13d-4b6a-8147-ada839348d63","Type":"ContainerStarted","Data":"a960dfaa41633016485595d72e7760355f6f00b8627a56dd61617772f44a30df"} Jul 24 22:35:39.266081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.266053 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" event={"ID":"847ae444-8ecf-4e2d-85a2-7e42057e472e","Type":"ContainerStarted","Data":"eeb832e9b95fdf769cbeeb1ff9293681e3b36b57f53bdf109497435a3b6989fa"} Jul 24 22:35:39.269323 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.269278 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-shbtf" event={"ID":"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4","Type":"ContainerStarted","Data":"17c1bed6416f3d05ba1df2ba1e53ebd9024a09f4f85e486b9c925d7e4ef6c7fa"} Jul 24 22:35:39.283524 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.283473 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5wqk" event={"ID":"b0936fc0-4e09-4ec8-9759-183233c4020f","Type":"ContainerStarted","Data":"4766db73cbbe8253c4a27d9acaa8228bd88ebbe63ce9af182ff6ef08ee04b078"} Jul 24 22:35:39.288656 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.288630 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kv4dv" event={"ID":"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178","Type":"ContainerStarted","Data":"619d057a71ba77010a3b4729e640ba8da994a8bedd42a06a6f899647f2f8d365"} Jul 24 22:35:39.299163 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.299136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" event={"ID":"6b2c4ffd-d9f2-459b-b1b2-290086b28121","Type":"ContainerStarted","Data":"d6c64fb9e4381dd3877eeb85f37f0bee2f7edea9fe48320bf658d87ffb752053"} Jul 24 22:35:39.335293 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.335229 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerStarted","Data":"dbe753ff952a8ef8b0b055099e9a68373f95eb2e6c8cd9270a9c2d9be3a284ff"} Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.830850 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.830976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:39.831018 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831167 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831234 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:35:41.831211435 +0000 UTC m=+5.038450226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831472 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831492 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831527 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831576 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:35:41.831560899 +0000 UTC m=+5.038799690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831661 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:39.831729 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:39.831697 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:35:41.8316859 +0000 UTC m=+5.038924693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:40.208362 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:40.208267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:40.208822 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:40.208448 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:40.208902 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:40.208870 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:40.208996 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:40.208976 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:40.209059 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:40.209052 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:40.209182 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:40.209127 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:40.225492 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:40.225452 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:30:38 +0000 UTC" deadline="2028-03-14 17:09:02.234225071 +0000 UTC" Jul 24 22:35:40.225492 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:40.225491 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14370h33m22.008739326s" Jul 24 22:35:41.849918 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:41.849860 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:41.850369 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:41.849939 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:41.850369 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:41.849980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:41.850369 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850169 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:41.850369 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850242 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:35:45.85022155 +0000 UTC m=+9.057460360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850725 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850785 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:35:45.85076665 +0000 UTC m=+9.058005456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850873 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850889 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850902 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:41.850976 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:41.850941 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:35:45.850928382 +0000 UTC m=+9.058167186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:42.208850 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:42.208248 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:42.208850 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:42.208407 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:42.208850 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:42.208776 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:42.209167 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:42.208776 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:42.209167 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:42.208887 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:42.209167 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:42.208978 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:44.208373 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:44.208518 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:44.208948 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:44.209055 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:44.209108 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:44.209236 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:44.209182 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:45.887507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:45.887474 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:45.887507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:45.887520 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:45.888145 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:45.887556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:45.888252 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888204 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:45.888414 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888281 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:35:53.888259023 +0000 UTC m=+17.095497828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:45.888414 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888314 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:45.888414 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888355 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:35:53.888344073 +0000 UTC m=+17.095582870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:45.888414 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888380 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:45.888414 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888408 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:45.888698 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888424 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:45.888698 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:45.888472 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:35:53.888459827 +0000 UTC m=+17.095698630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:46.208374 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:46.208222 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:46.208547 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:46.208402 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:46.208547 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:46.208483 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:46.209335 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:46.209305 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:46.209550 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:46.209523 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:46.214095 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:46.214053 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:48.207607 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:48.207572 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:48.208095 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:48.207705 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:48.208095 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:48.207826 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:48.208095 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:48.208008 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:48.208303 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:48.208137 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:48.208303 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:48.208255 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:50.208083 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:50.208043 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:50.208538 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:50.208075 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:50.208538 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:50.208171 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:50.208538 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:50.208220 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:50.208538 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:50.208340 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:50.208538 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:50.208416 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:52.208036 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:52.208008 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:52.208494 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:52.208015 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:52.208494 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:52.208123 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:52.208494 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:52.208138 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:52.208494 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:52.208272 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:52.208494 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:52.208370 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:53.953767 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:53.953735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:53.953767 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:53.953782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:53.953815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953896 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953910 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953910 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953931 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953947 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953953 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:36:09.953935041 +0000 UTC m=+33.161173841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953967 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:36:09.953959587 +0000 UTC m=+33.161198373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:35:53.954243 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:53.953990 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:09.953979006 +0000 UTC m=+33.161217806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:35:54.207745 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:54.207668 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:54.207745 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:54.207687 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:54.207990 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:54.207806 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:54.207990 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:54.207874 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:54.207990 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:54.207976 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:54.208160 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:54.208052 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:56.208138 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:56.208105 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:56.208595 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:56.208105 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:56.208595 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:56.208250 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:56.208595 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:56.208106 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:56.208595 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:56.208343 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:56.208595 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:56.208435 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:58.207401 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:58.207374 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:35:58.207401 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:58.207408 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:35:58.207903 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:58.207406 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:35:58.207903 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:58.207487 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:35:58.207903 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:58.207597 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:35:58.207903 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:35:58.207693 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:35:59.377278 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.377249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"9fc15d0561f4bef14189cafe930f9e8be89317f4fc077bfd5b6147327a127592"} Jul 24 22:35:59.378744 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.378722 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" event={"ID":"6b2c4ffd-d9f2-459b-b1b2-290086b28121","Type":"ContainerStarted","Data":"eaf6546589e0bc78406a4c5d14e76600e3cf4812b13c8e41922d52f79cbf896f"} Jul 24 22:35:59.380072 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.380052 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cgnnz" event={"ID":"5e4b8fc2-3980-4526-8d47-c519c7920505","Type":"ContainerStarted","Data":"0df53a5e3c291bab48c71544c7561277c315be7a64196c5cfab7d92bb215f3b6"} Jul 24 22:35:59.381378 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.381360 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" event={"ID":"542ff756433833167701e75caa9f94dd","Type":"ContainerStarted","Data":"350aade5cb048b8d211f70f6d01995e628dbb1de9f792ede50153d85b14e0b7e"} Jul 24 22:35:59.396312 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.396248 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wvsh9" podStartSLOduration=1.327558717 podStartE2EDuration="21.396214363s" totalImagesPullingTime="20.068655646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:35:59.395695531 +0000 UTC m=+22.602934350" watchObservedRunningTime="2026-07-24 22:35:59.396214363 +0000 UTC m=+22.603453171" Jul 24 22:35:59.411748 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.411717 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-123.ec2.internal" podStartSLOduration=21.411708611999998 podStartE2EDuration="21.411708612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:35:59.411594143 +0000 UTC m=+22.618832950" watchObservedRunningTime="2026-07-24 22:35:59.411708612 +0000 UTC m=+22.618947456" Jul 24 22:35:59.441761 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:35:59.441602 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cgnnz" podStartSLOduration=2.248027334 podStartE2EDuration="22.44158972s" totalImagesPullingTime="20.193562386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:35:59.441453848 +0000 UTC m=+22.648692683" watchObservedRunningTime="2026-07-24 22:35:59.44158972 +0000 UTC m=+22.648828526" Jul 24 22:36:00.208393 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.208161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:00.208541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.208177 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:00.208541 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:00.208437 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:00.208541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.208214 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:00.208640 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:00.208561 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:00.208640 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:00.208499 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:00.384389 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.384349 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5wqk" event={"ID":"b0936fc0-4e09-4ec8-9759-183233c4020f","Type":"ContainerStarted","Data":"a9b02e9c60df0ec64b53d18b732c7db099638c082a57b1267835fe9276c87fd6"} Jul 24 22:36:00.385748 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.385721 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kv4dv" event={"ID":"5f4f9dfe-beef-4b50-a0e3-62f8d61e0178","Type":"ContainerStarted","Data":"8c49c41eea3f98893ba5e22f062fe7464ea9777e6d494e7e18f36042cdc3ddd8"} Jul 24 22:36:00.387108 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.387083 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="67d35ea494d9e9e0d9124e0517cb921b3560dd7fcef504e46cbf6e07051abcec" exitCode=0 Jul 24 22:36:00.387213 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.387164 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"67d35ea494d9e9e0d9124e0517cb921b3560dd7fcef504e46cbf6e07051abcec"} Jul 24 22:36:00.388763 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.388741 2559 generic.go:356] "Generic (PLEG): container finished" podID="6e7e8d9d8bde8f18f73bb1ee2be3e58e" containerID="684c29c90c51234fd317b811535cf5c8e72bc392e348146384667a926785c6f5" exitCode=0 Jul 24 22:36:00.388870 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.388822 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" event={"ID":"6e7e8d9d8bde8f18f73bb1ee2be3e58e","Type":"ContainerDied","Data":"684c29c90c51234fd317b811535cf5c8e72bc392e348146384667a926785c6f5"} Jul 24 22:36:00.391542 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.391525 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:36:00.391817 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.391793 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f947503-67bd-4d12-88c7-9bf939ed72b6" containerID="87697f08913dbe4815f4d542f3cb731cf546123ff9275f34c932f3360d4d954c" exitCode=1 Jul 24 22:36:00.391894 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.391871 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"a0c27372c4650058f0cd1e8d77a4b210b6bbd7b767b398a8d942f601b7fc261d"} Jul 24 22:36:00.391944 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.391931 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"e677477774e4aa3340942bd1755fa642e2aac1c8872c5ef0f32a2a611939b9f6"} Jul 24 22:36:00.391996 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.391973 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"9d34058b4b9ca4b34a261a7d70ac5f6b7c3ab53e6ca7c0aa6d30c31e79ee3a48"} Jul 24 22:36:00.392052 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.392008 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"cadab9debff5d65ceea0abd51738756eee5afa1f35e77c9fcef652bc1dbd88f4"} Jul 24 22:36:00.392052 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.392041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerDied","Data":"87697f08913dbe4815f4d542f3cb731cf546123ff9275f34c932f3360d4d954c"} Jul 24 22:36:00.393335 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.393305 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6hxl" event={"ID":"65d397ae-e13d-4b6a-8147-ada839348d63","Type":"ContainerStarted","Data":"cc2c0a2c480aae36114fe27c39461e1f3659353949499726327fd50e0f7d332a"} Jul 24 22:36:00.394866 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.394847 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" event={"ID":"847ae444-8ecf-4e2d-85a2-7e42057e472e","Type":"ContainerStarted","Data":"b6f9c755e8ef8604d763e3180660c119cafae6b0cd9d230e4134b79819974fee"} Jul 24 22:36:00.396067 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.396047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-shbtf" event={"ID":"8cde2a49-cf6d-440a-9b1b-98f9539f7ef4","Type":"ContainerStarted","Data":"812f2141dc78ff27d1f05783a95eb36bdb3aa38a89e7121f91b6c3c4a82c529d"} Jul 24 22:36:00.400128 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.400098 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v5wqk" podStartSLOduration=2.361403587 podStartE2EDuration="22.400087192s" totalImagesPullingTime="20.038683605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:00.399733671 +0000 UTC m=+23.606972477" watchObservedRunningTime="2026-07-24 22:36:00.400087192 +0000 UTC m=+23.607326000" Jul 24 22:36:00.414487 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.414442 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kv4dv" podStartSLOduration=2.37094295 podStartE2EDuration="22.414425314s" totalImagesPullingTime="20.043482364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:00.413859475 +0000 UTC m=+23.621098281" watchObservedRunningTime="2026-07-24 22:36:00.414425314 +0000 UTC m=+23.621664122" Jul 24 22:36:00.470667 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.470633 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-shbtf" podStartSLOduration=2.353143002 podStartE2EDuration="22.470620284s" totalImagesPullingTime="20.117477282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:00.470320405 +0000 UTC m=+23.677559224" watchObservedRunningTime="2026-07-24 22:36:00.470620284 +0000 UTC m=+23.677859091" Jul 24 22:36:00.487414 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.487378 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m6hxl" podStartSLOduration=2.387563174 podStartE2EDuration="22.487365803s" totalImagesPullingTime="20.099802629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:00.487105595 +0000 UTC m=+23.694344412" watchObservedRunningTime="2026-07-24 22:36:00.487365803 +0000 UTC m=+23.694604612" Jul 24 22:36:00.673273 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:00.673252 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 22:36:01.181089 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.180952 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T22:36:00.673268005Z","UUID":"7aab7388-2752-4084-a3e1-758dbd46ab1b","Handler":null,"Name":"","Endpoint":""} Jul 24 22:36:01.182816 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.182791 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 22:36:01.182958 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.182835 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 22:36:01.400523 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.400489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" event={"ID":"847ae444-8ecf-4e2d-85a2-7e42057e472e","Type":"ContainerStarted","Data":"222d734d419d4a40f9b2a60f5f7879af56b662ece331bd6abfe8bae4ad0be60e"} Jul 24 22:36:01.402277 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.402246 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" event={"ID":"6e7e8d9d8bde8f18f73bb1ee2be3e58e","Type":"ContainerStarted","Data":"57966ccda9ac4b9063db5041e8991a664a8f81f6fc8428a47b5ef500e6ca77c2"} Jul 24 22:36:01.420344 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:01.420282 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-123.ec2.internal" podStartSLOduration=23.420264492 podStartE2EDuration="23.420264492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:01.420049739 +0000 UTC m=+24.627288548" watchObservedRunningTime="2026-07-24 22:36:01.420264492 +0000 UTC m=+24.627503300" Jul 24 22:36:02.208431 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.208407 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:02.208564 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.208407 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:02.208564 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:02.208524 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:02.208658 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:02.208636 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:02.208703 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.208407 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:02.208816 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:02.208792 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:02.408288 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.408060 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:36:02.408738 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.408676 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"cda6fb894880787d9385f4eb99d379c2aa836d543d6339b9fad8368ec203f553"} Jul 24 22:36:02.410857 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.410822 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" event={"ID":"847ae444-8ecf-4e2d-85a2-7e42057e472e","Type":"ContainerStarted","Data":"286bcc1a6f5db0c5c990da6c3869565a80f63bd38b9bcaa0e96a365adf52dd06"} Jul 24 22:36:02.934758 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.934715 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:36:02.935771 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.935749 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:36:02.952282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:02.952241 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8qht" podStartSLOduration=2.753795573 podStartE2EDuration="24.952210685s" totalImagesPullingTime="22.198415112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:02.437218235 +0000 UTC m=+25.644457042" watchObservedRunningTime="2026-07-24 22:36:02.952210685 +0000 UTC m=+26.159449492" Jul 24 22:36:03.413694 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:03.413665 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:36:03.414106 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:03.414030 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-shbtf" Jul 24 22:36:04.207380 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:04.207358 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:04.207481 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:04.207393 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:04.207481 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:04.207403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:04.207481 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:04.207467 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:04.207629 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:04.207514 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:04.207629 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:04.207615 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:05.420740 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.420561 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:36:05.421541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.421105 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"7fca483eb2e4cdce06bed44fb7422eb284e73b6772db609b5a24ecc0b6e05f23"} Jul 24 22:36:05.421541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.421344 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:05.421648 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.421567 2559 scope.go:117] "RemoveContainer" containerID="87697f08913dbe4815f4d542f3cb731cf546123ff9275f34c932f3360d4d954c" Jul 24 22:36:05.423603 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.423099 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="7858e19882c4bbd0eb53544b0da2e78ceaa4f1a4db43d4d627698b8421346032" exitCode=0 Jul 24 22:36:05.423603 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.423177 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"7858e19882c4bbd0eb53544b0da2e78ceaa4f1a4db43d4d627698b8421346032"} Jul 24 22:36:05.439947 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:05.439920 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:06.208262 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.208219 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:06.208480 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.208270 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:06.208480 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.208269 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:06.208480 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.208342 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:06.208480 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.208419 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:06.208636 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.208506 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:06.429733 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.429714 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:36:06.430105 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.430085 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" event={"ID":"4f947503-67bd-4d12-88c7-9bf939ed72b6","Type":"ContainerStarted","Data":"a7bb4322f546833ac914cec2361b413a40547c84ef0b08a778734f68c1e1e183"} Jul 24 22:36:06.430336 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.430315 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:06.430408 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.430379 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:06.445111 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.445090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:06.476281 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.476186 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" podStartSLOduration=7.783612339 podStartE2EDuration="28.476173331s" totalImagesPullingTime="20.692560992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:06.474437939 +0000 UTC m=+29.681676748" watchObservedRunningTime="2026-07-24 22:36:06.476173331 +0000 UTC m=+29.683412138" Jul 24 22:36:06.613191 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.613161 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bvd5v"] Jul 24 22:36:06.613356 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.613286 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:06.613397 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.613378 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:06.620431 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.620406 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mfq5d"] Jul 24 22:36:06.620551 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.620493 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:06.620657 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.620638 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:06.634244 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.634209 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpb69"] Jul 24 22:36:06.634373 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:06.634320 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:06.634427 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:06.634386 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:07.434349 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:07.434320 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="7092cb9e6c8129f975d81ae07aed8d1c9ff98cb98ed4bfb3587e2838bebe1585" exitCode=0 Jul 24 22:36:07.434717 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:07.434370 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"7092cb9e6c8129f975d81ae07aed8d1c9ff98cb98ed4bfb3587e2838bebe1585"} Jul 24 22:36:08.207655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:08.207616 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:08.207832 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:08.207727 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:08.207832 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:08.207738 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:08.207832 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:08.207731 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:08.207832 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:08.207814 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:08.207995 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:08.207918 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:09.444304 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:09.444273 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="034ab381023794b5cac747ec13639819fe0d420a344b378c075326a459db0da3" exitCode=0 Jul 24 22:36:09.444723 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:09.444314 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"034ab381023794b5cac747ec13639819fe0d420a344b378c075326a459db0da3"} Jul 24 22:36:09.976858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:09.976625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:09.976879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:09.976905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.976773 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.976992 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret podName:8d839f85-78a3-4588-b9d7-b7de549c7d3d nodeName:}" failed. No retries permitted until 2026-07-24 22:36:41.976971933 +0000 UTC m=+65.184210723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret") pod "global-pull-secret-syncer-mfq5d" (UID: "8d839f85-78a3-4588-b9d7-b7de549c7d3d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977035 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:36:09.977053 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977048 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:36:09.977362 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977072 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:36:09.977362 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977084 2559 projected.go:196] Error preparing data for projected volume kube-api-access-7wbrk for pod openshift-network-diagnostics/network-check-target-vpb69: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:36:09.977362 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977091 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs podName:ad0f0815-c320-4476-8580-cc6f0504457f nodeName:}" failed. No retries permitted until 2026-07-24 22:36:41.977075755 +0000 UTC m=+65.184314555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs") pod "network-metrics-daemon-bvd5v" (UID: "ad0f0815-c320-4476-8580-cc6f0504457f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:36:09.977362 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:09.977134 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk podName:73c86055-03f1-410b-8aca-973242d08c98 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:41.977118729 +0000 UTC m=+65.184357527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7wbrk" (UniqueName: "kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk") pod "network-check-target-vpb69" (UID: "73c86055-03f1-410b-8aca-973242d08c98") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:36:10.208350 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:10.208318 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:10.208350 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:10.208342 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:10.208586 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:10.208437 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:10.208586 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:10.208526 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:10.208682 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:10.208582 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:10.208730 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:10.208678 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:12.208416 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.208376 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:12.208856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.208383 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:12.208856 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.208516 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bvd5v" podUID="ad0f0815-c320-4476-8580-cc6f0504457f" Jul 24 22:36:12.208856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.208395 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:12.208856 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.208608 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mfq5d" podUID="8d839f85-78a3-4588-b9d7-b7de549c7d3d" Jul 24 22:36:12.208856 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.208707 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vpb69" podUID="73c86055-03f1-410b-8aca-973242d08c98" Jul 24 22:36:12.615775 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.615744 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-123.ec2.internal" event="NodeReady" Jul 24 22:36:12.615948 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.615902 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 22:36:12.667254 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.664672 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm"] Jul 24 22:36:12.670405 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.670381 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns"] Jul 24 22:36:12.670579 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.670512 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.673422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.673394 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 24 22:36:12.673541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.673497 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4lw4j\"" Jul 24 22:36:12.673609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.673585 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.673867 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.673849 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.674196 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.674175 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:36:12.674313 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.674296 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" Jul 24 22:36:12.676772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.676753 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.676925 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.676908 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.677007 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.676960 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kbft9\"" Jul 24 22:36:12.677645 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.677630 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qdl87"] Jul 24 22:36:12.677776 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.677762 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.681064 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.681035 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46"] Jul 24 22:36:12.681064 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.681046 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 22:36:12.681185 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.681152 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.682314 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.682099 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 22:36:12.682603 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.682584 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 22:36:12.688100 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.687786 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sxtln\"" Jul 24 22:36:12.690407 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.689863 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hdltk\"" Jul 24 22:36:12.690407 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.690111 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.690407 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.690392 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 24 22:36:12.693413 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.692401 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bdgq6"] Jul 24 22:36:12.696207 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.696189 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq"] Jul 24 22:36:12.696350 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.696334 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.697487 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.697467 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 22:36:12.698072 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.697874 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.698072 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.697963 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 24 22:36:12.698338 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.698318 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.700423 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.699702 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk"] Jul 24 22:36:12.700423 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.699744 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 24 22:36:12.700423 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.700073 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" Jul 24 22:36:12.701387 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.701370 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 24 22:36:12.701504 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.701485 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 24 22:36:12.702035 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.702016 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.702146 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.702128 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nw8k8\"" Jul 24 22:36:12.702912 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.702894 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-68bcdfbddf-x2qc8"] Jul 24 22:36:12.702912 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.702907 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.703126 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.703111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.708497 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.706310 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.708497 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.707158 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 24 22:36:12.708497 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.707478 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.708497 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.707557 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2"] Jul 24 22:36:12.708748 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.708593 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 24 22:36:12.711043 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.711024 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq"] Jul 24 22:36:12.711142 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.711054 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns"] Jul 24 22:36:12.711142 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.711070 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mdjzv"] Jul 24 22:36:12.711142 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.711079 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:12.714533 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.714516 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h2chv"] Jul 24 22:36:12.715274 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.715210 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:12.715496 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.715476 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 24 22:36:12.717016 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.716998 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.717131 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717006 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.717557 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717541 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm"] Jul 24 22:36:12.717696 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717685 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk"] Jul 24 22:36:12.717799 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717782 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46"] Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717808 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2"] Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717735 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717808 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717829 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qdl87"] Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717876 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.717893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717892 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:36:12.718173 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717898 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kqqgl\"" Jul 24 22:36:12.718173 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717785 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wgk62\"" Jul 24 22:36:12.718173 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.717635 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:12.718516 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718501 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 24 22:36:12.718628 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718510 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 24 22:36:12.718689 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718511 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.718740 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718515 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 24 22:36:12.718740 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718554 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xl56q\"" Jul 24 22:36:12.718837 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.718558 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.719924 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.719900 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5rbv8\"" Jul 24 22:36:12.721066 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.721026 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 24 22:36:12.721334 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.721318 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 22:36:12.722370 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.722349 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 22:36:12.722900 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.722882 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lqkr5\"" Jul 24 22:36:12.723985 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.723969 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 24 22:36:12.724109 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.724088 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bdgq6"] Jul 24 22:36:12.724337 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.724296 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mdjzv"] Jul 24 22:36:12.724548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.724533 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 24 22:36:12.724855 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.724837 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.725244 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.725206 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt4n5\"" Jul 24 22:36:12.726088 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.726072 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.727153 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.727137 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 22:36:12.727502 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.727488 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.727830 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.727815 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 24 22:36:12.727994 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.727972 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:12.728115 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.727927 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 24 22:36:12.730856 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.730835 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qh7d5\"" Jul 24 22:36:12.733102 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.733083 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h2chv"] Jul 24 22:36:12.755178 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.755155 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68bcdfbddf-x2qc8"] Jul 24 22:36:12.797327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797338 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-config\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.797446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797362 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-service-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhhp7\" (UniqueName: \"kubernetes.io/projected/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-kube-api-access-fhhp7\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hr9h\" (UniqueName: \"kubernetes.io/projected/2edd2a28-e797-4584-a305-a81fae76f1d0-kube-api-access-8hr9h\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.797573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-trusted-ca\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.797573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797495 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.797573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797525 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-tmp\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-serving-cert\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797743 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797577 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.797743 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797631 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca775d01-4dc6-409e-bd30-dc3337e7f028-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.797743 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797671 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x65jk\" (UniqueName: \"kubernetes.io/projected/ca775d01-4dc6-409e-bd30-dc3337e7f028-kube-api-access-x65jk\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.797743 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797698 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-snapshots\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.797743 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797729 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pw48\" (UniqueName: \"kubernetes.io/projected/2ea42247-dd62-400c-8121-43c5c0d48750-kube-api-access-6pw48\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.797905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.797905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797783 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlx5h\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.797905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797832 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.797905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-stats-auth\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.797905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.798080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797913 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.798080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797942 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.798080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.797982 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf52s\" (UniqueName: \"kubernetes.io/projected/a5a49982-99c8-43df-82be-1d74ecf06753-kube-api-access-cf52s\") pod \"volume-data-source-validator-549c6d54cf-swsns\" (UID: \"a5a49982-99c8-43df-82be-1d74ecf06753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" Jul 24 22:36:12.798080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798009 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmc6z\" (UniqueName: \"kubernetes.io/projected/40b96aa5-e3d4-4c8f-bff6-a29518db9eb9-kube-api-access-zmc6z\") pod \"network-check-source-7d88cb55d8-hc4nq\" (UID: \"40b96aa5-e3d4-4c8f-bff6-a29518db9eb9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" Jul 24 22:36:12.798080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798028 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69dea1e1-26cb-408d-b7db-23f1aeed1250-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798101 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ss5h\" (UniqueName: \"kubernetes.io/projected/22f63a83-bc41-41ae-b1ca-6d860473d275-kube-api-access-9ss5h\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798153 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69dea1e1-26cb-408d-b7db-23f1aeed1250-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bv4\" (UniqueName: \"kubernetes.io/projected/69dea1e1-26cb-408d-b7db-23f1aeed1250-kube-api-access-p6bv4\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798240 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f63a83-bc41-41ae-b1ca-6d860473d275-serving-cert\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.798319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798300 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-default-certificate\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.798555 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798336 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.798555 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798385 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.798555 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.798424 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.899310 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899277 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hr9h\" (UniqueName: \"kubernetes.io/projected/2edd2a28-e797-4584-a305-a81fae76f1d0-kube-api-access-8hr9h\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.899310 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899311 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-trusted-ca\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-tmp\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899386 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-serving-cert\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899423 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca775d01-4dc6-409e-bd30-dc3337e7f028-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.899456 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.899474 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:12.899545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899473 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x65jk\" (UniqueName: \"kubernetes.io/projected/ca775d01-4dc6-409e-bd30-dc3337e7f028-kube-api-access-x65jk\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.899585 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.399562712 +0000 UTC m=+36.606801514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899668 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-snapshots\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pw48\" (UniqueName: \"kubernetes.io/projected/2ea42247-dd62-400c-8121-43c5c0d48750-kube-api-access-6pw48\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899749 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlx5h\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899811 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdb02f3c-8fca-4daa-b79b-63e8456559ea-cert\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899853 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm4nh\" (UniqueName: \"kubernetes.io/projected/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-kube-api-access-wm4nh\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899877 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899881 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-tmp\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-stats-auth\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.899921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899927 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899950 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.899975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20729795-56c7-49cc-934c-d9df6f309413-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.899990 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900006 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf52s\" (UniqueName: \"kubernetes.io/projected/a5a49982-99c8-43df-82be-1d74ecf06753-kube-api-access-cf52s\") pod \"volume-data-source-validator-549c6d54cf-swsns\" (UID: \"a5a49982-99c8-43df-82be-1d74ecf06753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.900087 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.400031052 +0000 UTC m=+36.607269838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900129 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmc6z\" (UniqueName: \"kubernetes.io/projected/40b96aa5-e3d4-4c8f-bff6-a29518db9eb9-kube-api-access-zmc6z\") pod \"network-check-source-7d88cb55d8-hc4nq\" (UID: \"40b96aa5-e3d4-4c8f-bff6-a29518db9eb9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69dea1e1-26cb-408d-b7db-23f1aeed1250-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-tmp-dir\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900304 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20729795-56c7-49cc-934c-d9df6f309413-config\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900313 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-snapshots\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900317 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca775d01-4dc6-409e-bd30-dc3337e7f028-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900354 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ss5h\" (UniqueName: \"kubernetes.io/projected/22f63a83-bc41-41ae-b1ca-6d860473d275-kube-api-access-9ss5h\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900381 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69dea1e1-26cb-408d-b7db-23f1aeed1250-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.900479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900401 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2dxl\" (UniqueName: \"kubernetes.io/projected/cdb02f3c-8fca-4daa-b79b-63e8456559ea-kube-api-access-f2dxl\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900420 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgbb\" (UniqueName: \"kubernetes.io/projected/20729795-56c7-49cc-934c-d9df6f309413-kube-api-access-jqgbb\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900474 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-trusted-ca\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.900489 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.900539 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.400526217 +0000 UTC m=+36.607765025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900678 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900767 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/69dea1e1-26cb-408d-b7db-23f1aeed1250-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900850 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.900877 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bv4\" (UniqueName: \"kubernetes.io/projected/69dea1e1-26cb-408d-b7db-23f1aeed1250-kube-api-access-p6bv4\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.900960 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.901084 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.40106995 +0000 UTC m=+36.608308750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901157 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f63a83-bc41-41ae-b1ca-6d860473d275-serving-cert\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901180 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-default-certificate\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901200 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.901285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901238 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901307 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-config\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-config-volume\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-service-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901390 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhhp7\" (UniqueName: \"kubernetes.io/projected/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-kube-api-access-fhhp7\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.901431 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.902081 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:12.901936 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.401921853 +0000 UTC m=+36.609160658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:12.902450 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.902174 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/22f63a83-bc41-41ae-b1ca-6d860473d275-config\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.903386 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.902679 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-service-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.903386 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.903079 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.903548 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.903466 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.905101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905074 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-stats-auth\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.905101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905079 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/22f63a83-bc41-41ae-b1ca-6d860473d275-serving-cert\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.905484 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905438 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-serving-cert\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.905573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905505 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.905573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905508 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/69dea1e1-26cb-408d-b7db-23f1aeed1250-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.905573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905554 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-default-certificate\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.905872 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.905847 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.909032 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.909008 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hr9h\" (UniqueName: \"kubernetes.io/projected/2edd2a28-e797-4584-a305-a81fae76f1d0-kube-api-access-8hr9h\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:12.909812 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.909787 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x65jk\" (UniqueName: \"kubernetes.io/projected/ca775d01-4dc6-409e-bd30-dc3337e7f028-kube-api-access-x65jk\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:12.911752 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.911716 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlx5h\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.912148 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.912129 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmc6z\" (UniqueName: \"kubernetes.io/projected/40b96aa5-e3d4-4c8f-bff6-a29518db9eb9-kube-api-access-zmc6z\") pod \"network-check-source-7d88cb55d8-hc4nq\" (UID: \"40b96aa5-e3d4-4c8f-bff6-a29518db9eb9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" Jul 24 22:36:12.912346 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.912253 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:12.913722 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.913703 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ss5h\" (UniqueName: \"kubernetes.io/projected/22f63a83-bc41-41ae-b1ca-6d860473d275-kube-api-access-9ss5h\") pod \"console-operator-584dcb6b54-bdgq6\" (UID: \"22f63a83-bc41-41ae-b1ca-6d860473d275\") " pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:12.914134 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.914117 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf52s\" (UniqueName: \"kubernetes.io/projected/a5a49982-99c8-43df-82be-1d74ecf06753-kube-api-access-cf52s\") pod \"volume-data-source-validator-549c6d54cf-swsns\" (UID: \"a5a49982-99c8-43df-82be-1d74ecf06753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" Jul 24 22:36:12.914511 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.914487 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bv4\" (UniqueName: \"kubernetes.io/projected/69dea1e1-26cb-408d-b7db-23f1aeed1250-kube-api-access-p6bv4\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-npd46\" (UID: \"69dea1e1-26cb-408d-b7db-23f1aeed1250\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:12.916841 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.916793 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhhp7\" (UniqueName: \"kubernetes.io/projected/40f6c541-38fd-4e28-8d80-80f2fa2c7cb3-kube-api-access-fhhp7\") pod \"insights-operator-5885fdb759-qdl87\" (UID: \"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3\") " pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:12.916841 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.916802 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pw48\" (UniqueName: \"kubernetes.io/projected/2ea42247-dd62-400c-8121-43c5c0d48750-kube-api-access-6pw48\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:12.952163 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.952083 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:12.998779 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:12.998714 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" Jul 24 22:36:13.002025 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.001999 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20729795-56c7-49cc-934c-d9df6f309413-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.002152 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-tmp-dir\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.002205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002163 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20729795-56c7-49cc-934c-d9df6f309413-config\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.002284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002208 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2dxl\" (UniqueName: \"kubernetes.io/projected/cdb02f3c-8fca-4daa-b79b-63e8456559ea-kube-api-access-f2dxl\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:13.002284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgbb\" (UniqueName: \"kubernetes.io/projected/20729795-56c7-49cc-934c-d9df6f309413-kube-api-access-jqgbb\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.002284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002275 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.002427 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-config-volume\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.002427 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002413 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdb02f3c-8fca-4daa-b79b-63e8456559ea-cert\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:13.002516 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm4nh\" (UniqueName: \"kubernetes.io/projected/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-kube-api-access-wm4nh\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.002516 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002475 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-tmp-dir\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.002606 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.002576 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:13.002658 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.002632 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:13.502613987 +0000 UTC m=+36.709852785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:13.003016 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.002991 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/20729795-56c7-49cc-934c-d9df6f309413-config\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.003141 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.003068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-config-volume\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.004998 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.004977 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/20729795-56c7-49cc-934c-d9df6f309413-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.005326 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.005307 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdb02f3c-8fca-4daa-b79b-63e8456559ea-cert\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:13.011977 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.011954 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgbb\" (UniqueName: \"kubernetes.io/projected/20729795-56c7-49cc-934c-d9df6f309413-kube-api-access-jqgbb\") pod \"service-ca-operator-5d6c75f84f-qtxj2\" (UID: \"20729795-56c7-49cc-934c-d9df6f309413\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.012703 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.012684 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2dxl\" (UniqueName: \"kubernetes.io/projected/cdb02f3c-8fca-4daa-b79b-63e8456559ea-kube-api-access-f2dxl\") pod \"ingress-canary-h2chv\" (UID: \"cdb02f3c-8fca-4daa-b79b-63e8456559ea\") " pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:13.013991 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.013970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm4nh\" (UniqueName: \"kubernetes.io/projected/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-kube-api-access-wm4nh\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.016842 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.016826 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-qdl87" Jul 24 22:36:13.035169 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.035140 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:13.036213 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.036196 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" Jul 24 22:36:13.043361 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.043345 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" Jul 24 22:36:13.052346 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.052325 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:13.065299 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.065283 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" Jul 24 22:36:13.078147 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.078132 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h2chv" Jul 24 22:36:13.278012 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.277904 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:13.383201 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.382781 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.408121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.408189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.408336 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.408392 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.408445 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.408654 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.408671 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.408762 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.408741378 +0000 UTC m=+37.615980177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409192 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409268 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.409250383 +0000 UTC m=+37.616489170 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409326 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409345 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.409330963 +0000 UTC m=+37.616569749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409362 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.409353358 +0000 UTC m=+37.616592146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409409 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:13.421800 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.409444 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.409432565 +0000 UTC m=+37.616671351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:13.510076 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.509819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:13.510202 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.510009 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:13.510202 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:13.510186 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:14.510166309 +0000 UTC m=+37.717405108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:13.567326 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.567298 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46"] Jul 24 22:36:13.570697 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.570673 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dea1e1_26cb_408d_b7db_23f1aeed1250.slice/crio-d307d508935dcd95b18e11707cdde391f168136b21369eb2f6254a1759cc6031 WatchSource:0}: Error finding container d307d508935dcd95b18e11707cdde391f168136b21369eb2f6254a1759cc6031: Status 404 returned error can't find the container with id d307d508935dcd95b18e11707cdde391f168136b21369eb2f6254a1759cc6031 Jul 24 22:36:13.575446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.575422 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2"] Jul 24 22:36:13.577537 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.577516 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20729795_56c7_49cc_934c_d9df6f309413.slice/crio-52dc290beb65cad8c871beff9e2eb8c8b487b66e609065811bd537b600f42616 WatchSource:0}: Error finding container 52dc290beb65cad8c871beff9e2eb8c8b487b66e609065811bd537b600f42616: Status 404 returned error can't find the container with id 52dc290beb65cad8c871beff9e2eb8c8b487b66e609065811bd537b600f42616 Jul 24 22:36:13.579066 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.579047 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-bdgq6"] Jul 24 22:36:13.584190 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.584153 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22f63a83_bc41_41ae_b1ca_6d860473d275.slice/crio-b5dbd967687b8d828e9866b39210fdb2a3543572707cb8cc44a9a5bdaa4e19ff WatchSource:0}: Error finding container b5dbd967687b8d828e9866b39210fdb2a3543572707cb8cc44a9a5bdaa4e19ff: Status 404 returned error can't find the container with id b5dbd967687b8d828e9866b39210fdb2a3543572707cb8cc44a9a5bdaa4e19ff Jul 24 22:36:13.586700 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.586677 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qdl87"] Jul 24 22:36:13.591905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.590450 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq"] Jul 24 22:36:13.591905 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.591735 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40f6c541_38fd_4e28_8d80_80f2fa2c7cb3.slice/crio-33676a6c523b9d78ec30fa851bf3b0f09ef2b2f1033df4478ca83b9e7a6093a4 WatchSource:0}: Error finding container 33676a6c523b9d78ec30fa851bf3b0f09ef2b2f1033df4478ca83b9e7a6093a4: Status 404 returned error can't find the container with id 33676a6c523b9d78ec30fa851bf3b0f09ef2b2f1033df4478ca83b9e7a6093a4 Jul 24 22:36:13.593037 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.592919 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h2chv"] Jul 24 22:36:13.603480 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:13.603459 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns"] Jul 24 22:36:13.606045 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.606022 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb02f3c_8fca_4daa_b79b_63e8456559ea.slice/crio-0a0b2fb532f948e6f1abaafd2a11be00043b479bfed961884a4d171238a9df47 WatchSource:0}: Error finding container 0a0b2fb532f948e6f1abaafd2a11be00043b479bfed961884a4d171238a9df47: Status 404 returned error can't find the container with id 0a0b2fb532f948e6f1abaafd2a11be00043b479bfed961884a4d171238a9df47 Jul 24 22:36:13.607498 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.607474 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5a49982_99c8_43df_82be_1d74ecf06753.slice/crio-a9676060c65ab285a52f2abb056b97599e830f73a5318b9697accaf0a090c9ba WatchSource:0}: Error finding container a9676060c65ab285a52f2abb056b97599e830f73a5318b9697accaf0a090c9ba: Status 404 returned error can't find the container with id a9676060c65ab285a52f2abb056b97599e830f73a5318b9697accaf0a090c9ba Jul 24 22:36:13.607937 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:13.607918 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b96aa5_e3d4_4c8f_bff6_a29518db9eb9.slice/crio-887aaa277b0d16eaa475457e91ac52117f34810543158ed3b9feb404243e7f19 WatchSource:0}: Error finding container 887aaa277b0d16eaa475457e91ac52117f34810543158ed3b9feb404243e7f19: Status 404 returned error can't find the container with id 887aaa277b0d16eaa475457e91ac52117f34810543158ed3b9feb404243e7f19 Jul 24 22:36:14.208283 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.208246 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:14.208770 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.208754 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:14.212285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.209272 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:14.216062 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.214439 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ncjs\"" Jul 24 22:36:14.216062 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.214681 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 22:36:14.216062 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.214872 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29mp5\"" Jul 24 22:36:14.216062 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.215029 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 22:36:14.418398 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.418358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.418434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.418477 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.418537 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.418574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.418703 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.418766 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.418747619 +0000 UTC m=+39.625986420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419187 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:14.419268 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419257 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.419242425 +0000 UTC m=+39.626481223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:14.419745 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419728 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.41971238 +0000 UTC m=+39.626951168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:14.419864 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419815 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:14.419864 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419827 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:14.420018 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419878 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.419866111 +0000 UTC m=+39.627104912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:14.420018 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419939 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:14.420018 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.419975 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.419964333 +0000 UTC m=+39.627203124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:14.470642 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.470495 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" event={"ID":"a5a49982-99c8-43df-82be-1d74ecf06753","Type":"ContainerStarted","Data":"a9676060c65ab285a52f2abb056b97599e830f73a5318b9697accaf0a090c9ba"} Jul 24 22:36:14.478603 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.478571 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qdl87" event={"ID":"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3","Type":"ContainerStarted","Data":"33676a6c523b9d78ec30fa851bf3b0f09ef2b2f1033df4478ca83b9e7a6093a4"} Jul 24 22:36:14.488699 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.488549 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="898104acecd57f0d141e925088377e86f06cd8dc19701d1c9b538603821168fc" exitCode=0 Jul 24 22:36:14.488699 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.488642 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"898104acecd57f0d141e925088377e86f06cd8dc19701d1c9b538603821168fc"} Jul 24 22:36:14.492446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.490897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" event={"ID":"20729795-56c7-49cc-934c-d9df6f309413","Type":"ContainerStarted","Data":"52dc290beb65cad8c871beff9e2eb8c8b487b66e609065811bd537b600f42616"} Jul 24 22:36:14.496056 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.496031 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" event={"ID":"69dea1e1-26cb-408d-b7db-23f1aeed1250","Type":"ContainerStarted","Data":"d307d508935dcd95b18e11707cdde391f168136b21369eb2f6254a1759cc6031"} Jul 24 22:36:14.500507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.498650 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" event={"ID":"40b96aa5-e3d4-4c8f-bff6-a29518db9eb9","Type":"ContainerStarted","Data":"887aaa277b0d16eaa475457e91ac52117f34810543158ed3b9feb404243e7f19"} Jul 24 22:36:14.501707 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.501435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h2chv" event={"ID":"cdb02f3c-8fca-4daa-b79b-63e8456559ea","Type":"ContainerStarted","Data":"0a0b2fb532f948e6f1abaafd2a11be00043b479bfed961884a4d171238a9df47"} Jul 24 22:36:14.503398 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.503372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" event={"ID":"22f63a83-bc41-41ae-b1ca-6d860473d275","Type":"ContainerStarted","Data":"b5dbd967687b8d828e9866b39210fdb2a3543572707cb8cc44a9a5bdaa4e19ff"} Jul 24 22:36:14.523737 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:14.519898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:14.523737 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.520058 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:14.523737 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:14.520121 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:16.520099842 +0000 UTC m=+39.727338637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:15.522131 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:15.521246 2559 generic.go:356] "Generic (PLEG): container finished" podID="acb8d463-3918-4645-afd2-26855ea8f5dc" containerID="aa0be29b6e332d19c8d0bf3388e25559e9d409948cdfb0d24c11471647277a69" exitCode=0 Jul 24 22:36:15.522131 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:15.521300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerDied","Data":"aa0be29b6e332d19c8d0bf3388e25559e9d409948cdfb0d24c11471647277a69"} Jul 24 22:36:16.091439 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.091181 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:16.191590 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.191550 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" limit=1024 Jul 24 22:36:16.444939 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.444853 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:16.444939 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.444931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:16.445104 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.444980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:16.445104 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.445047 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:16.445104 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.445086 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:16.445199 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445186 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:16.445278 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445264 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.445244644 +0000 UTC m=+43.652483449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:16.445368 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445325 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.445311619 +0000 UTC m=+43.652550404 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:16.445422 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445387 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:16.445422 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445387 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:16.445422 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445420 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.445410612 +0000 UTC m=+43.652649401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:16.445545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445464 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.445443744 +0000 UTC m=+43.652682555 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:16.445545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445387 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:16.445545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445489 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:16.445545 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.445544 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.445528565 +0000 UTC m=+43.652767369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:16.527640 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.527602 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" event={"ID":"acb8d463-3918-4645-afd2-26855ea8f5dc","Type":"ContainerStarted","Data":"3e449054eb2952506e2de8b39e8301a84e4981734cec63d2492489fbb7ba6a66"} Jul 24 22:36:16.545848 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.545815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:16.545981 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.545960 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:16.546056 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:16.546028 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:20.546008049 +0000 UTC m=+43.753246841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:16.556969 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:16.556926 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jhbt8" podStartSLOduration=9.246976547 podStartE2EDuration="39.556900073s" totalImagesPullingTime="30.309923526s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:16.554619647 +0000 UTC m=+39.761858453" watchObservedRunningTime="2026-07-24 22:36:16.556900073 +0000 UTC m=+39.764138880" Jul 24 22:36:20.483908 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.483855 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.483934 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.483986 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484053 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.484033302 +0000 UTC m=+51.691272111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484059 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484071 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.483987 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484103 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.484092731 +0000 UTC m=+51.691331530 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484128 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.484120071 +0000 UTC m=+51.691358860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.484193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.484266 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484310 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484362 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484370 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.484352878 +0000 UTC m=+51.691591679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:20.484529 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.484396 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.484385624 +0000 UTC m=+51.691624412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:20.585130 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:20.585098 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:20.585334 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.585243 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:20.585334 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:20.585302 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:28.585286666 +0000 UTC m=+51.792525457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:21.544713 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.544677 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" event={"ID":"40b96aa5-e3d4-4c8f-bff6-a29518db9eb9","Type":"ContainerStarted","Data":"b5d68ceb1c65c77f665c3dcefca795b63bf419073a38c506937d7e1acd30c949"} Jul 24 22:36:21.546505 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.546477 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h2chv" event={"ID":"cdb02f3c-8fca-4daa-b79b-63e8456559ea","Type":"ContainerStarted","Data":"51b6ff6c504cb7aa420661d49d2b90b4e0330311be670ca8ea0489651a9eebb2"} Jul 24 22:36:21.549296 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.549269 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" event={"ID":"22f63a83-bc41-41ae-b1ca-6d860473d275","Type":"ContainerStarted","Data":"945d6c3f8f4366c67827234779f70bc20289fffbe16f863d40033dcb63047122"} Jul 24 22:36:21.550081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.550057 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:21.551732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.551501 2559 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-bdgq6 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body= Jul 24 22:36:21.551732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.551550 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" podUID="22f63a83-bc41-41ae-b1ca-6d860473d275" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" Jul 24 22:36:21.557580 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.557178 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" event={"ID":"a5a49982-99c8-43df-82be-1d74ecf06753","Type":"ContainerStarted","Data":"1f7cfb2ba8cb1a23b1847ec1080d2e80f58e8a53c48ccfb7a2ac74d3bceebc1f"} Jul 24 22:36:21.560432 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.559709 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qdl87" event={"ID":"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3","Type":"ContainerStarted","Data":"57a14940df702e6f9fcefcd102d79afaca96fff4cbdaf2702adfec7973ec1939"} Jul 24 22:36:21.562707 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.562126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" event={"ID":"20729795-56c7-49cc-934c-d9df6f309413","Type":"ContainerStarted","Data":"7bbc464562121adc0540b35a0f8775d786c5cde3da7f50daddafe1c6f53247c6"} Jul 24 22:36:21.564414 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.564347 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" event={"ID":"69dea1e1-26cb-408d-b7db-23f1aeed1250","Type":"ContainerStarted","Data":"0d3eacb55cd3eec2c7b44fb8bbcd3f55d56c4a062ea24be26f5ac6e678cd2455"} Jul 24 22:36:21.567938 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.567735 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h2chv" podStartSLOduration=1.900253953 podStartE2EDuration="9.567716742s" totalImagesPullingTime="7.667462789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.567680919 +0000 UTC m=+44.774919753" watchObservedRunningTime="2026-07-24 22:36:21.567716742 +0000 UTC m=+44.774955548" Jul 24 22:36:21.619438 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.619381 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-swsns" podStartSLOduration=16.069362113 podStartE2EDuration="23.619334754s" totalImagesPullingTime="7.549972641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.597017214 +0000 UTC m=+44.804256033" watchObservedRunningTime="2026-07-24 22:36:21.619334754 +0000 UTC m=+44.826573563" Jul 24 22:36:21.646434 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.646355 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-qdl87" podStartSLOduration=25.964586122 podStartE2EDuration="33.646334712s" totalImagesPullingTime="7.68174859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.620458401 +0000 UTC m=+44.827697190" watchObservedRunningTime="2026-07-24 22:36:21.646334712 +0000 UTC m=+44.853573521" Jul 24 22:36:21.646757 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.646720 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" podStartSLOduration=15.960249154 podStartE2EDuration="23.646708651s" totalImagesPullingTime="7.686459497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.646643514 +0000 UTC m=+44.853882323" watchObservedRunningTime="2026-07-24 22:36:21.646708651 +0000 UTC m=+44.853947439" Jul 24 22:36:21.665172 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.665125 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" podStartSLOduration=15.963621129 podStartE2EDuration="23.665111978s" totalImagesPullingTime="7.701490849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.664614454 +0000 UTC m=+44.871853264" watchObservedRunningTime="2026-07-24 22:36:21.665111978 +0000 UTC m=+44.872350802" Jul 24 22:36:21.685134 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:21.685074 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" podStartSLOduration=15.989316626 podStartE2EDuration="23.685055776s" totalImagesPullingTime="7.69573915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:21.683439711 +0000 UTC m=+44.890678519" watchObservedRunningTime="2026-07-24 22:36:21.685055776 +0000 UTC m=+44.892294586" Jul 24 22:36:22.569440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.569411 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/0.log" Jul 24 22:36:22.569911 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.569460 2559 generic.go:356] "Generic (PLEG): container finished" podID="22f63a83-bc41-41ae-b1ca-6d860473d275" containerID="945d6c3f8f4366c67827234779f70bc20289fffbe16f863d40033dcb63047122" exitCode=255 Jul 24 22:36:22.569911 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.569608 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" event={"ID":"22f63a83-bc41-41ae-b1ca-6d860473d275","Type":"ContainerDied","Data":"945d6c3f8f4366c67827234779f70bc20289fffbe16f863d40033dcb63047122"} Jul 24 22:36:22.569911 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.569842 2559 scope.go:117] "RemoveContainer" containerID="945d6c3f8f4366c67827234779f70bc20289fffbe16f863d40033dcb63047122" Jul 24 22:36:22.620380 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.620341 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hc4nq" podStartSLOduration=3.945590603 podStartE2EDuration="11.620325337s" totalImagesPullingTime="7.674734734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:22.619914148 +0000 UTC m=+45.827152971" watchObservedRunningTime="2026-07-24 22:36:22.620325337 +0000 UTC m=+45.827564144" Jul 24 22:36:22.635594 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.635572 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq"] Jul 24 22:36:22.657725 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.657701 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq"] Jul 24 22:36:22.657831 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.657817 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" Jul 24 22:36:22.660822 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.660803 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 22:36:22.660926 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.660823 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 22:36:22.660926 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.660821 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g4vd6\"" Jul 24 22:36:22.708394 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.708353 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fffpq\" (UniqueName: \"kubernetes.io/projected/2b836276-d3af-4232-b7c2-d0c028c64e7e-kube-api-access-fffpq\") pod \"migrator-6865c9967-fmnxq\" (UID: \"2b836276-d3af-4232-b7c2-d0c028c64e7e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" Jul 24 22:36:22.809464 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.809430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fffpq\" (UniqueName: \"kubernetes.io/projected/2b836276-d3af-4232-b7c2-d0c028c64e7e-kube-api-access-fffpq\") pod \"migrator-6865c9967-fmnxq\" (UID: \"2b836276-d3af-4232-b7c2-d0c028c64e7e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" Jul 24 22:36:22.819654 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.819626 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fffpq\" (UniqueName: \"kubernetes.io/projected/2b836276-d3af-4232-b7c2-d0c028c64e7e-kube-api-access-fffpq\") pod \"migrator-6865c9967-fmnxq\" (UID: \"2b836276-d3af-4232-b7c2-d0c028c64e7e\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" Jul 24 22:36:22.971044 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:22.971010 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" Jul 24 22:36:23.035628 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.035604 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:23.121524 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.121459 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq"] Jul 24 22:36:23.123595 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:23.123567 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b836276_d3af_4232_b7c2_d0c028c64e7e.slice/crio-f541822e16f545eb3e624630890fb9bff0ea9409447d313e78d843f57922b118 WatchSource:0}: Error finding container f541822e16f545eb3e624630890fb9bff0ea9409447d313e78d843f57922b118: Status 404 returned error can't find the container with id f541822e16f545eb3e624630890fb9bff0ea9409447d313e78d843f57922b118 Jul 24 22:36:23.333679 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.333614 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bmgbj"] Jul 24 22:36:23.367511 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.367481 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bmgbj"] Jul 24 22:36:23.367671 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.367591 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.370338 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.370317 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 22:36:23.370472 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.370393 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xgn84\"" Jul 24 22:36:23.370578 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.370565 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 22:36:23.416527 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.416498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.416649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.416566 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/788fe0e1-ae93-41a9-a3be-2890ca680fd0-data-volume\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.416649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.416617 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwd9r\" (UniqueName: \"kubernetes.io/projected/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-api-access-bwd9r\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.416719 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.416677 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/788fe0e1-ae93-41a9-a3be-2890ca680fd0-crio-socket\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.416719 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.416710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.517942 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.517917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwd9r\" (UniqueName: \"kubernetes.io/projected/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-api-access-bwd9r\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.517974 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/788fe0e1-ae93-41a9-a3be-2890ca680fd0-crio-socket\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.518001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.518040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518272 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.518098 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/788fe0e1-ae93-41a9-a3be-2890ca680fd0-crio-socket\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518272 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:23.518108 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:36:23.518272 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:23.518115 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:36:23.518272 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:23.518168 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:24.018151281 +0000 UTC m=+47.225390096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : secret "insights-runtime-extractor-tls" not found Jul 24 22:36:23.518272 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:23.518187 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:24.018178068 +0000 UTC m=+47.225416855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : configmap "kube-rbac-proxy" not found Jul 24 22:36:23.518485 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.518279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/788fe0e1-ae93-41a9-a3be-2890ca680fd0-data-volume\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.518586 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.518571 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/788fe0e1-ae93-41a9-a3be-2890ca680fd0-data-volume\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.526951 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.526930 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwd9r\" (UniqueName: \"kubernetes.io/projected/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-api-access-bwd9r\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:23.574529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.574509 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:36:23.574968 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.574954 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/0.log" Jul 24 22:36:23.575010 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.574988 2559 generic.go:356] "Generic (PLEG): container finished" podID="22f63a83-bc41-41ae-b1ca-6d860473d275" containerID="e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6" exitCode=255 Jul 24 22:36:23.575059 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.575041 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" event={"ID":"22f63a83-bc41-41ae-b1ca-6d860473d275","Type":"ContainerDied","Data":"e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6"} Jul 24 22:36:23.575105 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.575093 2559 scope.go:117] "RemoveContainer" containerID="945d6c3f8f4366c67827234779f70bc20289fffbe16f863d40033dcb63047122" Jul 24 22:36:23.575295 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.575275 2559 scope.go:117] "RemoveContainer" containerID="e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6" Jul 24 22:36:23.575491 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:23.575469 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bdgq6_openshift-console-operator(22f63a83-bc41-41ae-b1ca-6d860473d275)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" podUID="22f63a83-bc41-41ae-b1ca-6d860473d275" Jul 24 22:36:23.576446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:23.576262 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" event={"ID":"2b836276-d3af-4232-b7c2-d0c028c64e7e","Type":"ContainerStarted","Data":"f541822e16f545eb3e624630890fb9bff0ea9409447d313e78d843f57922b118"} Jul 24 22:36:24.022671 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:24.022596 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:24.022840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:24.022678 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:24.022840 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:24.022737 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:36:24.022840 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:24.022813 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:25.022792277 +0000 UTC m=+48.230031062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : configmap "kube-rbac-proxy" not found Jul 24 22:36:24.022840 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:24.022831 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:36:24.023050 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:24.022885 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:25.022868847 +0000 UTC m=+48.230107637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : secret "insights-runtime-extractor-tls" not found Jul 24 22:36:24.580754 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:24.580725 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:36:24.581198 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:24.581040 2559 scope.go:117] "RemoveContainer" containerID="e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6" Jul 24 22:36:24.581285 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:24.581189 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bdgq6_openshift-console-operator(22f63a83-bc41-41ae-b1ca-6d860473d275)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" podUID="22f63a83-bc41-41ae-b1ca-6d860473d275" Jul 24 22:36:25.031472 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.031443 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:25.031595 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:25.031579 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:36:25.031647 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:25.031638 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:27.031622195 +0000 UTC m=+50.238860982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : secret "insights-runtime-extractor-tls" not found Jul 24 22:36:25.031687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.031669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:25.031751 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:25.031739 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:36:25.031786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:25.031781 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:27.031770402 +0000 UTC m=+50.239009194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : configmap "kube-rbac-proxy" not found Jul 24 22:36:25.521563 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.521534 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kv4dv_5f4f9dfe-beef-4b50-a0e3-62f8d61e0178/dns-node-resolver/0.log" Jul 24 22:36:25.586064 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.586025 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" event={"ID":"2b836276-d3af-4232-b7c2-d0c028c64e7e","Type":"ContainerStarted","Data":"4b4cebec2779606e69cc8989d972cd5f22c896ff3c704529593f70771713fef4"} Jul 24 22:36:25.586463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.586088 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" event={"ID":"2b836276-d3af-4232-b7c2-d0c028c64e7e","Type":"ContainerStarted","Data":"0363dc361ad3a81b98dc966c6ccbcb1a1c7a6827055d1aa6b1909a2ea67ccb83"} Jul 24 22:36:25.587211 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.587190 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6qvc"] Jul 24 22:36:25.605152 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.605109 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fmnxq" podStartSLOduration=1.927199603 podStartE2EDuration="3.605096186s" totalImagesPullingTime="1.677896583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:25.604083593 +0000 UTC m=+48.811322401" watchObservedRunningTime="2026-07-24 22:36:25.605096186 +0000 UTC m=+48.812335045" Jul 24 22:36:25.611421 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.611400 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6qvc"] Jul 24 22:36:25.611537 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.611494 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.614251 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.614217 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 22:36:25.614527 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.614509 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 22:36:25.614527 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.614519 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p8zcd\"" Jul 24 22:36:25.614651 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.614511 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 22:36:25.614651 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.614518 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 22:36:25.736999 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.736971 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-cabundle\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.737136 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.737037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpgcw\" (UniqueName: \"kubernetes.io/projected/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-kube-api-access-lpgcw\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.737136 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.737082 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-key\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.837986 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.837959 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-key\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.838105 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.838079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-cabundle\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.838161 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.838146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpgcw\" (UniqueName: \"kubernetes.io/projected/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-kube-api-access-lpgcw\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.838865 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.838766 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-cabundle\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.840560 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.840541 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-signing-key\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.850969 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.850946 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpgcw\" (UniqueName: \"kubernetes.io/projected/eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c-kube-api-access-lpgcw\") pod \"service-ca-658d7b7c78-h6qvc\" (UID: \"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c\") " pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:25.920192 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:25.920165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" Jul 24 22:36:26.052532 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.051376 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-h6qvc"] Jul 24 22:36:26.056643 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:26.056616 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2979a8_0eaf_48a8_b5e5_46d4a5052d8c.slice/crio-b7cd016e04f1176308f813d0e6981cd7e20342ce061cd6858853be99f49ad3ed WatchSource:0}: Error finding container b7cd016e04f1176308f813d0e6981cd7e20342ce061cd6858853be99f49ad3ed: Status 404 returned error can't find the container with id b7cd016e04f1176308f813d0e6981cd7e20342ce061cd6858853be99f49ad3ed Jul 24 22:36:26.521865 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.521793 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5wqk_b0936fc0-4e09-4ec8-9759-183233c4020f/node-ca/0.log" Jul 24 22:36:26.593324 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.593280 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" event={"ID":"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c","Type":"ContainerStarted","Data":"99d7766f30917fb0fb31bfd41f9f8aa9b4c666d4e33ad798268717aff01bed1f"} Jul 24 22:36:26.593771 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.593333 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" event={"ID":"eb2979a8-0eaf-48a8-b5e5-46d4a5052d8c","Type":"ContainerStarted","Data":"b7cd016e04f1176308f813d0e6981cd7e20342ce061cd6858853be99f49ad3ed"} Jul 24 22:36:26.620906 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.620857 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-h6qvc" podStartSLOduration=1.6208445679999999 podStartE2EDuration="1.620844568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:26.619215025 +0000 UTC m=+49.826453834" watchObservedRunningTime="2026-07-24 22:36:26.620844568 +0000 UTC m=+49.828083376" Jul 24 22:36:26.921304 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:26.921267 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h2chv_cdb02f3c-8fca-4daa-b79b-63e8456559ea/serve-healthcheck-canary/0.log" Jul 24 22:36:27.049015 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:27.048980 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:27.049188 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:27.049038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:27.049188 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:27.049098 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:36:27.049188 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:27.049153 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:31.049138267 +0000 UTC m=+54.256377052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : configmap "kube-rbac-proxy" not found Jul 24 22:36:27.049366 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:27.049199 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:36:27.049366 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:27.049268 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:31.049254344 +0000 UTC m=+54.256493133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : secret "insights-runtime-extractor-tls" not found Jul 24 22:36:27.124646 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:27.124614 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fmnxq_2b836276-d3af-4232-b7c2-d0c028c64e7e/migrator/0.log" Jul 24 22:36:27.322074 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:27.322011 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fmnxq_2b836276-d3af-4232-b7c2-d0c028c64e7e/graceful-termination/0.log" Jul 24 22:36:27.528428 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:27.528390 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-npd46_69dea1e1-26cb-408d-b7db-23f1aeed1250/kube-storage-version-migrator-operator/0.log" Jul 24 22:36:28.561320 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.561282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.561347 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.561412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561454 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561533 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls podName:ca775d01-4dc6-409e-bd30-dc3337e7f028 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.561510794 +0000 UTC m=+67.768749583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-6vdvk" (UID: "ca775d01-4dc6-409e-bd30-dc3337e7f028") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561551 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.561542877 +0000 UTC m=+67.768781668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : configmap references non-existent config key: service-ca.crt Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.561455 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561607 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.561625 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561644 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs podName:2edd2a28-e797-4584-a305-a81fae76f1d0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.561629979 +0000 UTC m=+67.768868771 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs") pod "router-default-68bcdfbddf-x2qc8" (UID: "2edd2a28-e797-4584-a305-a81fae76f1d0") : secret "router-metrics-certs-default" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561692 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561703 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79c8d7d4cd-df4fz: secret "image-registry-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561733 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls podName:c76426fe-f152-460b-b873-3d2931d1e7ac nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.561723175 +0000 UTC m=+67.768961964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls") pod "image-registry-79c8d7d4cd-df4fz" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac") : secret "image-registry-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561758 2559 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 22:36:28.561786 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.561786 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls podName:2ea42247-dd62-400c-8121-43c5c0d48750 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.561776977 +0000 UTC m=+67.769015770 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls") pod "cluster-samples-operator-6748489656-qrkgm" (UID: "2ea42247-dd62-400c-8121-43c5c0d48750") : secret "samples-operator-tls" not found Jul 24 22:36:28.662599 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:28.662559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:28.662784 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.662702 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:36:28.662784 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:28.662755 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls podName:74e5ce1f-4f12-49d5-9d35-542cbeb3bc34 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:44.662737967 +0000 UTC m=+67.869976759 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls") pod "dns-default-mdjzv" (UID: "74e5ce1f-4f12-49d5-9d35-542cbeb3bc34") : secret "dns-default-metrics-tls" not found Jul 24 22:36:31.084445 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:31.084412 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:31.084882 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:31.084524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:31.084882 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:31.084566 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:36:31.084882 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:31.084638 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls podName:788fe0e1-ae93-41a9-a3be-2890ca680fd0 nodeName:}" failed. No retries permitted until 2026-07-24 22:36:39.084622145 +0000 UTC m=+62.291860931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bmgbj" (UID: "788fe0e1-ae93-41a9-a3be-2890ca680fd0") : secret "insights-runtime-extractor-tls" not found Jul 24 22:36:31.085002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:31.084929 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/788fe0e1-ae93-41a9-a3be-2890ca680fd0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:31.550322 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:31.550280 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:31.550653 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:31.550639 2559 scope.go:117] "RemoveContainer" containerID="e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6" Jul 24 22:36:31.550796 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:36:31.550778 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-bdgq6_openshift-console-operator(22f63a83-bc41-41ae-b1ca-6d860473d275)\"" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" podUID="22f63a83-bc41-41ae-b1ca-6d860473d275" Jul 24 22:36:33.035879 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.035820 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:33.036353 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.036213 2559 scope.go:117] "RemoveContainer" containerID="e7f2be52ccd60d2d54c8b233c876378c22312dba64ef9f06e942f4f94f8073e6" Jul 24 22:36:33.617543 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.617518 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:36:33.617698 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.617562 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" event={"ID":"22f63a83-bc41-41ae-b1ca-6d860473d275","Type":"ContainerStarted","Data":"85bb9a649318e94b70ddb2112654a93d992cf3ac3e2f640fca0312212f52aa4d"} Jul 24 22:36:33.617873 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.617826 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:33.622139 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:33.622118 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-bdgq6" Jul 24 22:36:38.451002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:38.450969 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lplhc" Jul 24 22:36:39.154549 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.154503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:39.156855 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.156832 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/788fe0e1-ae93-41a9-a3be-2890ca680fd0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bmgbj\" (UID: \"788fe0e1-ae93-41a9-a3be-2890ca680fd0\") " pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:39.280397 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.280363 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xgn84\"" Jul 24 22:36:39.287802 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.287781 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bmgbj" Jul 24 22:36:39.411767 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.411563 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bmgbj"] Jul 24 22:36:39.414337 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:39.414312 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788fe0e1_ae93_41a9_a3be_2890ca680fd0.slice/crio-d0aee0cf272356e122fe6c163929a600b74802ef5b0503ec3d0f549ae29f0a43 WatchSource:0}: Error finding container d0aee0cf272356e122fe6c163929a600b74802ef5b0503ec3d0f549ae29f0a43: Status 404 returned error can't find the container with id d0aee0cf272356e122fe6c163929a600b74802ef5b0503ec3d0f549ae29f0a43 Jul 24 22:36:39.638272 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:39.638220 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmgbj" event={"ID":"788fe0e1-ae93-41a9-a3be-2890ca680fd0","Type":"ContainerStarted","Data":"d0aee0cf272356e122fe6c163929a600b74802ef5b0503ec3d0f549ae29f0a43"} Jul 24 22:36:41.982857 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.982823 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:41.982857 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.982865 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:41.983352 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.983055 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:41.985824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.985799 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 22:36:41.985957 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.985914 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wbrk\" (UniqueName: \"kubernetes.io/projected/73c86055-03f1-410b-8aca-973242d08c98-kube-api-access-7wbrk\") pod \"network-check-target-vpb69\" (UID: \"73c86055-03f1-410b-8aca-973242d08c98\") " pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:41.986026 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.986012 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 22:36:41.995799 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.995774 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d839f85-78a3-4588-b9d7-b7de549c7d3d-original-pull-secret\") pod \"global-pull-secret-syncer-mfq5d\" (UID: \"8d839f85-78a3-4588-b9d7-b7de549c7d3d\") " pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:41.998514 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:41.998490 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad0f0815-c320-4476-8580-cc6f0504457f-metrics-certs\") pod \"network-metrics-daemon-bvd5v\" (UID: \"ad0f0815-c320-4476-8580-cc6f0504457f\") " pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:42.130867 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:42.130839 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29mp5\"" Jul 24 22:36:42.138163 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:42.138139 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bvd5v" Jul 24 22:36:42.139573 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:42.139555 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5ncjs\"" Jul 24 22:36:42.143953 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:42.143932 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mfq5d" Jul 24 22:36:42.148066 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:42.148040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:44.274732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.274704 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vpb69"] Jul 24 22:36:44.277147 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:44.277117 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c86055_03f1_410b_8aca_973242d08c98.slice/crio-940280f081d9faf5cbf40376cb805fe32b81c78992b034dfe0b43a94c702c03b WatchSource:0}: Error finding container 940280f081d9faf5cbf40376cb805fe32b81c78992b034dfe0b43a94c702c03b: Status 404 returned error can't find the container with id 940280f081d9faf5cbf40376cb805fe32b81c78992b034dfe0b43a94c702c03b Jul 24 22:36:44.493209 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.493136 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mfq5d"] Jul 24 22:36:44.498111 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.498086 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bvd5v"] Jul 24 22:36:44.501072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:44.501050 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0f0815_c320_4476_8580_cc6f0504457f.slice/crio-7c785a9381911ffe741c2f84184c395892a17cde091fdb8c715602f7e4612c59 WatchSource:0}: Error finding container 7c785a9381911ffe741c2f84184c395892a17cde091fdb8c715602f7e4612c59: Status 404 returned error can't find the container with id 7c785a9381911ffe741c2f84184c395892a17cde091fdb8c715602f7e4612c59 Jul 24 22:36:44.605986 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.605965 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:44.606080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.606004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:44.606080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.606045 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:44.606080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.606078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:44.606252 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.606104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:44.607244 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.607197 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2edd2a28-e797-4584-a305-a81fae76f1d0-service-ca-bundle\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:44.608677 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.608651 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca775d01-4dc6-409e-bd30-dc3337e7f028-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-6vdvk\" (UID: \"ca775d01-4dc6-409e-bd30-dc3337e7f028\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:44.608779 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.608695 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2edd2a28-e797-4584-a305-a81fae76f1d0-metrics-certs\") pod \"router-default-68bcdfbddf-x2qc8\" (UID: \"2edd2a28-e797-4584-a305-a81fae76f1d0\") " pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:44.608823 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.608776 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"image-registry-79c8d7d4cd-df4fz\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:44.609107 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.609090 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ea42247-dd62-400c-8121-43c5c0d48750-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-qrkgm\" (UID: \"2ea42247-dd62-400c-8121-43c5c0d48750\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:44.655795 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.655763 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmgbj" event={"ID":"788fe0e1-ae93-41a9-a3be-2890ca680fd0","Type":"ContainerStarted","Data":"9874a81c754151d6f4966cdfc8dd52e2797a2209d58f1b6c03650a27303d30c8"} Jul 24 22:36:44.656934 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.656914 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvd5v" event={"ID":"ad0f0815-c320-4476-8580-cc6f0504457f","Type":"ContainerStarted","Data":"7c785a9381911ffe741c2f84184c395892a17cde091fdb8c715602f7e4612c59"} Jul 24 22:36:44.657991 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.657969 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mfq5d" event={"ID":"8d839f85-78a3-4588-b9d7-b7de549c7d3d","Type":"ContainerStarted","Data":"51df621a3c7d0db95cb26b92b5de203d520772b050819b7f26bbff236a0a9102"} Jul 24 22:36:44.659336 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.659318 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpb69" event={"ID":"73c86055-03f1-410b-8aca-973242d08c98","Type":"ContainerStarted","Data":"8a89d24b8dfc4491477bde102c3961fbce97229e88f238fa487e2fbf81140649"} Jul 24 22:36:44.659415 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.659350 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vpb69" event={"ID":"73c86055-03f1-410b-8aca-973242d08c98","Type":"ContainerStarted","Data":"940280f081d9faf5cbf40376cb805fe32b81c78992b034dfe0b43a94c702c03b"} Jul 24 22:36:44.659415 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.659398 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:36:44.678045 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.678005 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vpb69" podStartSLOduration=66.677993635 podStartE2EDuration="1m6.677993635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:44.676612962 +0000 UTC m=+67.883851770" watchObservedRunningTime="2026-07-24 22:36:44.677993635 +0000 UTC m=+67.885232441" Jul 24 22:36:44.706440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.706416 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:44.708640 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.708623 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/74e5ce1f-4f12-49d5-9d35-542cbeb3bc34-metrics-tls\") pod \"dns-default-mdjzv\" (UID: \"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34\") " pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:44.786561 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.786514 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4lw4j\"" Jul 24 22:36:44.794770 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.794752 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" Jul 24 22:36:44.810262 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.810247 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sxtln\"" Jul 24 22:36:44.818547 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.818528 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:44.854031 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.854010 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xl56q\"" Jul 24 22:36:44.860940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.860876 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-5rbv8\"" Jul 24 22:36:44.860940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.860909 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" Jul 24 22:36:44.869266 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.869203 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:44.875490 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.875465 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bt4n5\"" Jul 24 22:36:44.884023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.883914 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:44.965839 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:44.963438 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm"] Jul 24 22:36:45.030765 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.030724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:36:45.101257 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.097790 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk"] Jul 24 22:36:45.106176 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:45.106145 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca775d01_4dc6_409e_bd30_dc3337e7f028.slice/crio-9e92ec48aa270da05a99d6aa1f12ee71d8974ccaebade5f04415321231c75536 WatchSource:0}: Error finding container 9e92ec48aa270da05a99d6aa1f12ee71d8974ccaebade5f04415321231c75536: Status 404 returned error can't find the container with id 9e92ec48aa270da05a99d6aa1f12ee71d8974ccaebade5f04415321231c75536 Jul 24 22:36:45.135504 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.135479 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-68bcdfbddf-x2qc8"] Jul 24 22:36:45.137771 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:45.137746 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edd2a28_e797_4584_a305_a81fae76f1d0.slice/crio-4a37d2cf1dff600dc4f6f028fdf4d06766721e6bdd741ed4d73a4bb21f11ad1c WatchSource:0}: Error finding container 4a37d2cf1dff600dc4f6f028fdf4d06766721e6bdd741ed4d73a4bb21f11ad1c: Status 404 returned error can't find the container with id 4a37d2cf1dff600dc4f6f028fdf4d06766721e6bdd741ed4d73a4bb21f11ad1c Jul 24 22:36:45.149430 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.149382 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mdjzv"] Jul 24 22:36:45.153730 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:45.153703 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e5ce1f_4f12_49d5_9d35_542cbeb3bc34.slice/crio-725243d7a78aa1793380fd5866c9594e35e40ac3dce76e8c256083afa067898f WatchSource:0}: Error finding container 725243d7a78aa1793380fd5866c9594e35e40ac3dce76e8c256083afa067898f: Status 404 returned error can't find the container with id 725243d7a78aa1793380fd5866c9594e35e40ac3dce76e8c256083afa067898f Jul 24 22:36:45.668031 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.667998 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" event={"ID":"2ea42247-dd62-400c-8121-43c5c0d48750","Type":"ContainerStarted","Data":"cf9e9a8d96c731a4cbc2b3f7189ca3f6ab304b7e2f73142fae276ca87ee1c950"} Jul 24 22:36:45.669802 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.669755 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" event={"ID":"2edd2a28-e797-4584-a305-a81fae76f1d0","Type":"ContainerStarted","Data":"f361ca5a4764429881e1ee548e76f7d43ed2c63240d819949ff32527ff809d9e"} Jul 24 22:36:45.669802 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.669804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" event={"ID":"2edd2a28-e797-4584-a305-a81fae76f1d0","Type":"ContainerStarted","Data":"4a37d2cf1dff600dc4f6f028fdf4d06766721e6bdd741ed4d73a4bb21f11ad1c"} Jul 24 22:36:45.672437 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.672367 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" event={"ID":"ca775d01-4dc6-409e-bd30-dc3337e7f028","Type":"ContainerStarted","Data":"9e92ec48aa270da05a99d6aa1f12ee71d8974ccaebade5f04415321231c75536"} Jul 24 22:36:45.674167 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.674104 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mdjzv" event={"ID":"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34","Type":"ContainerStarted","Data":"725243d7a78aa1793380fd5866c9594e35e40ac3dce76e8c256083afa067898f"} Jul 24 22:36:45.678628 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.678604 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" event={"ID":"c76426fe-f152-460b-b873-3d2931d1e7ac","Type":"ContainerStarted","Data":"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906"} Jul 24 22:36:45.678713 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.678645 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" event={"ID":"c76426fe-f152-460b-b873-3d2931d1e7ac","Type":"ContainerStarted","Data":"caea038cc30648452054c447f807dd9f6f438872b60b9e60f03fcac90b17b7a1"} Jul 24 22:36:45.678713 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.678681 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:36:45.690651 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.690286 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" podStartSLOduration=57.690267279 podStartE2EDuration="57.690267279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:45.688677902 +0000 UTC m=+68.895916731" watchObservedRunningTime="2026-07-24 22:36:45.690267279 +0000 UTC m=+68.897506087" Jul 24 22:36:45.869527 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.869502 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:45.876609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.876575 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:45.901522 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:45.901070 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" podStartSLOduration=67.901050359 podStartE2EDuration="1m7.901050359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:45.70979019 +0000 UTC m=+68.917028997" watchObservedRunningTime="2026-07-24 22:36:45.901050359 +0000 UTC m=+69.108289167" Jul 24 22:36:46.478811 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.478778 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-p79sp"] Jul 24 22:36:46.482293 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.481117 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.482293 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.481969 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t"] Jul 24 22:36:46.485257 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.484956 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 24 22:36:46.485257 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.485135 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx"] Jul 24 22:36:46.485257 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.485215 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 24 22:36:46.486663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.485816 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.488152 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.487022 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.488555 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-h9zzr\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.488866 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.489058 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.489131 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.489263 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-27cmp\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.489394 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 22:36:46.490023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.489901 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 24 22:36:46.496924 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.494539 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2"] Jul 24 22:36:46.498967 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.498380 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.503346 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.500187 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t"] Jul 24 22:36:46.503346 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.501727 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx"] Jul 24 22:36:46.503346 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.503087 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-p79sp"] Jul 24 22:36:46.512665 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.508937 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 22:36:46.512665 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.509186 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 22:36:46.512665 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.509415 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 22:36:46.512665 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.509640 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 22:36:46.512981 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.512966 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2"] Jul 24 22:36:46.586597 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.586561 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-cwnrf"] Jul 24 22:36:46.589754 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.589679 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:36:46.593629 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.592981 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hrz5b\"" Jul 24 22:36:46.593629 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.593374 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 22:36:46.593629 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.593481 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 22:36:46.601660 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.601591 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-cwnrf"] Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.621983 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8c079b5-0c68-4a74-a210-4a91ad677ea9-tmp\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622024 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1441bb85-82b6-482c-ab37-713eda4c2a79-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622059 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622131 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fktgd\" (UniqueName: \"kubernetes.io/projected/e8c079b5-0c68-4a74-a210-4a91ad677ea9-kube-api-access-fktgd\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e8c079b5-0c68-4a74-a210-4a91ad677ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d378d4b-73a5-4839-8a5e-febd01167e24-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622320 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622351 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622399 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1d378d4b-73a5-4839-8a5e-febd01167e24-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622431 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622457 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwkg2\" (UniqueName: \"kubernetes.io/projected/1441bb85-82b6-482c-ab37-713eda4c2a79-kube-api-access-jwkg2\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.622657 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.622485 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j29c6\" (UniqueName: \"kubernetes.io/projected/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-kube-api-access-j29c6\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.688282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.688249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvd5v" event={"ID":"ad0f0815-c320-4476-8580-cc6f0504457f","Type":"ContainerStarted","Data":"325bdb849bd013b3d205f150640398b865221928ace8db6ff62270df6f44c39b"} Jul 24 22:36:46.688734 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.688718 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bvd5v" event={"ID":"ad0f0815-c320-4476-8580-cc6f0504457f","Type":"ContainerStarted","Data":"f8eccdc44a00ee8378d7954182ae946eaabf0dda388b8f140f882a991cd1796b"} Jul 24 22:36:46.690110 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.690061 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:46.691464 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.691420 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-68bcdfbddf-x2qc8" Jul 24 22:36:46.713498 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.713396 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bvd5v" podStartSLOduration=67.433874556 podStartE2EDuration="1m8.713377475s" totalImagesPullingTime="1.279502919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:46.710270368 +0000 UTC m=+69.917509175" watchObservedRunningTime="2026-07-24 22:36:46.713377475 +0000 UTC m=+69.920616285" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723113 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1d378d4b-73a5-4839-8a5e-febd01167e24-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723154 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwkg2\" (UniqueName: \"kubernetes.io/projected/1441bb85-82b6-482c-ab37-713eda4c2a79-kube-api-access-jwkg2\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723207 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j29c6\" (UniqueName: \"kubernetes.io/projected/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-kube-api-access-j29c6\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8c079b5-0c68-4a74-a210-4a91ad677ea9-tmp\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.723695 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.723281 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1441bb85-82b6-482c-ab37-713eda4c2a79-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724134 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fktgd\" (UniqueName: \"kubernetes.io/projected/e8c079b5-0c68-4a74-a210-4a91ad677ea9-kube-api-access-fktgd\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724258 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtmkf\" (UniqueName: \"kubernetes.io/projected/435b83b2-108b-4edb-becb-c7c3702db1d2-kube-api-access-wtmkf\") pod \"downloads-7c97dc4555-cwnrf\" (UID: \"435b83b2-108b-4edb-becb-c7c3702db1d2\") " pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e8c079b5-0c68-4a74-a210-4a91ad677ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724329 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d378d4b-73a5-4839-8a5e-febd01167e24-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724326 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1441bb85-82b6-482c-ab37-713eda4c2a79-additional-service-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.724499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724408 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.724904 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724536 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.724904 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.724574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.725636 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.725597 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8c079b5-0c68-4a74-a210-4a91ad677ea9-tmp\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.725990 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.725966 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1d378d4b-73a5-4839-8a5e-febd01167e24-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.738378 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.729438 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.738378 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.729970 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.738378 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.734893 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.738378 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.738361 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1441bb85-82b6-482c-ab37-713eda4c2a79-ca\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.738595 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.738574 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.739300 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.739219 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1d378d4b-73a5-4839-8a5e-febd01167e24-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-p79sp\" (UID: \"1d378d4b-73a5-4839-8a5e-febd01167e24\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.739754 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.739700 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e8c079b5-0c68-4a74-a210-4a91ad677ea9-klusterlet-config\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.742329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.742249 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fktgd\" (UniqueName: \"kubernetes.io/projected/e8c079b5-0c68-4a74-a210-4a91ad677ea9-kube-api-access-fktgd\") pod \"klusterlet-addon-workmgr-5bd56b6d5f-zxvfx\" (UID: \"e8c079b5-0c68-4a74-a210-4a91ad677ea9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.742424 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.742408 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwkg2\" (UniqueName: \"kubernetes.io/projected/1441bb85-82b6-482c-ab37-713eda4c2a79-kube-api-access-jwkg2\") pod \"cluster-proxy-proxy-agent-74f7b8f8fc-xhls2\" (UID: \"1441bb85-82b6-482c-ab37-713eda4c2a79\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.745347 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.745306 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j29c6\" (UniqueName: \"kubernetes.io/projected/b6cc603d-dc69-4a7a-b97d-0d89b4414ff0-kube-api-access-j29c6\") pod \"managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t\" (UID: \"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.801119 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.801086 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" Jul 24 22:36:46.820355 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.820088 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" Jul 24 22:36:46.827706 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.825373 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtmkf\" (UniqueName: \"kubernetes.io/projected/435b83b2-108b-4edb-becb-c7c3702db1d2-kube-api-access-wtmkf\") pod \"downloads-7c97dc4555-cwnrf\" (UID: \"435b83b2-108b-4edb-becb-c7c3702db1d2\") " pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:36:46.830040 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.829872 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:36:46.840275 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.840117 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" Jul 24 22:36:46.852329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.851993 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtmkf\" (UniqueName: \"kubernetes.io/projected/435b83b2-108b-4edb-becb-c7c3702db1d2-kube-api-access-wtmkf\") pod \"downloads-7c97dc4555-cwnrf\" (UID: \"435b83b2-108b-4edb-becb-c7c3702db1d2\") " pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:36:46.912493 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:46.912256 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:36:47.032447 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.031916 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-p79sp"] Jul 24 22:36:47.061502 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.061479 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t"] Jul 24 22:36:47.071410 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:47.071381 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6cc603d_dc69_4a7a_b97d_0d89b4414ff0.slice/crio-bef287efdffaa3548bc2c163af01f83b553dff131dd94b3f8259e17f10686753 WatchSource:0}: Error finding container bef287efdffaa3548bc2c163af01f83b553dff131dd94b3f8259e17f10686753: Status 404 returned error can't find the container with id bef287efdffaa3548bc2c163af01f83b553dff131dd94b3f8259e17f10686753 Jul 24 22:36:47.126953 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.126880 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx"] Jul 24 22:36:47.133187 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:47.133047 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c079b5_0c68_4a74_a210_4a91ad677ea9.slice/crio-cb2999e17f158fe583fe0f86483880e9836f7a7474c6a879da3edc8fcb155e59 WatchSource:0}: Error finding container cb2999e17f158fe583fe0f86483880e9836f7a7474c6a879da3edc8fcb155e59: Status 404 returned error can't find the container with id cb2999e17f158fe583fe0f86483880e9836f7a7474c6a879da3edc8fcb155e59 Jul 24 22:36:47.162772 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.162135 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2"] Jul 24 22:36:47.171618 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:36:47.171588 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1441bb85_82b6_482c_ab37_713eda4c2a79.slice/crio-00a0e190e0c5ba1adbdc54b56e6d2f061321ea2e33687d823281dc5a687e2441 WatchSource:0}: Error finding container 00a0e190e0c5ba1adbdc54b56e6d2f061321ea2e33687d823281dc5a687e2441: Status 404 returned error can't find the container with id 00a0e190e0c5ba1adbdc54b56e6d2f061321ea2e33687d823281dc5a687e2441 Jul 24 22:36:47.181718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.181358 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-cwnrf"] Jul 24 22:36:47.694812 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.694715 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" event={"ID":"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0","Type":"ContainerStarted","Data":"bef287efdffaa3548bc2c163af01f83b553dff131dd94b3f8259e17f10686753"} Jul 24 22:36:47.696520 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.696489 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-cwnrf" event={"ID":"435b83b2-108b-4edb-becb-c7c3702db1d2","Type":"ContainerStarted","Data":"8d4ab857d1f52dfc3bef4c50b2f8664f3ec9b6bf96aacc639fd5229e46819e23"} Jul 24 22:36:47.698070 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.698038 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" event={"ID":"1d378d4b-73a5-4839-8a5e-febd01167e24","Type":"ContainerStarted","Data":"d8df333fa7178445e4c32ea6ab091a7fa16743a1c4f774c985f2a3e844eb1b50"} Jul 24 22:36:47.699462 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.699436 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" event={"ID":"e8c079b5-0c68-4a74-a210-4a91ad677ea9","Type":"ContainerStarted","Data":"cb2999e17f158fe583fe0f86483880e9836f7a7474c6a879da3edc8fcb155e59"} Jul 24 22:36:47.700840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:47.700815 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" event={"ID":"1441bb85-82b6-482c-ab37-713eda4c2a79","Type":"ContainerStarted","Data":"00a0e190e0c5ba1adbdc54b56e6d2f061321ea2e33687d823281dc5a687e2441"} Jul 24 22:36:55.741220 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.741179 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" event={"ID":"1d378d4b-73a5-4839-8a5e-febd01167e24","Type":"ContainerStarted","Data":"b031b517bfd0849bea007f4cd34b1877eeeb60349ada420c8abe6d05c30db606"} Jul 24 22:36:55.742952 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.742925 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mfq5d" event={"ID":"8d839f85-78a3-4588-b9d7-b7de549c7d3d","Type":"ContainerStarted","Data":"7e12f113992be4f5434c6836b231ffd66f183c39b19b2cae2a73bd02654d9b1d"} Jul 24 22:36:55.745456 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.745423 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" event={"ID":"2ea42247-dd62-400c-8121-43c5c0d48750","Type":"ContainerStarted","Data":"43d364ab8573fbed8dea1cb779d319823b92bbb3a6373d3e2b04b7190fcf2ab2"} Jul 24 22:36:55.745587 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.745478 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" event={"ID":"2ea42247-dd62-400c-8121-43c5c0d48750","Type":"ContainerStarted","Data":"4673cdc6837fac6d22648ac4bfbc9a4458dc6b51ea1ac5d3341802df6c8d03a0"} Jul 24 22:36:55.747609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.747584 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmgbj" event={"ID":"788fe0e1-ae93-41a9-a3be-2890ca680fd0","Type":"ContainerStarted","Data":"4cb772f0156b2c4482592e9008ee3b8dc5cdc12349c63af6a1d8d29d630584ec"} Jul 24 22:36:55.749239 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.749174 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" event={"ID":"ca775d01-4dc6-409e-bd30-dc3337e7f028","Type":"ContainerStarted","Data":"68b3af3429e6dce4ccc88590e6dc84d529de71a13ed1ee3c772e262580612429"} Jul 24 22:36:55.751171 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.751114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mdjzv" event={"ID":"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34","Type":"ContainerStarted","Data":"4e834137e931b802a9947b18e0af375032f14e362d358f6d1cb69fa5c0567fdb"} Jul 24 22:36:55.751171 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.751158 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mdjzv" event={"ID":"74e5ce1f-4f12-49d5-9d35-542cbeb3bc34","Type":"ContainerStarted","Data":"e18832b3611afa797ea19c970449f51f76220177b02b93c943f588919bea6dcf"} Jul 24 22:36:55.751418 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.751265 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mdjzv" Jul 24 22:36:55.759062 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.758760 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-p79sp" podStartSLOduration=1.490201571 podStartE2EDuration="9.758743478s" totalImagesPullingTime="8.268541907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:55.756986832 +0000 UTC m=+78.964225636" watchObservedRunningTime="2026-07-24 22:36:55.758743478 +0000 UTC m=+78.965982286" Jul 24 22:36:55.776512 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.776465 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mdjzv" podStartSLOduration=34.226462929 podStartE2EDuration="43.776447682s" totalImagesPullingTime="9.549984753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:55.775322935 +0000 UTC m=+78.982561763" watchObservedRunningTime="2026-07-24 22:36:55.776447682 +0000 UTC m=+78.983686513" Jul 24 22:36:55.792162 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.792108 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-6vdvk" podStartSLOduration=58.195210893 podStartE2EDuration="1m7.792089607s" totalImagesPullingTime="9.596878714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:55.791452815 +0000 UTC m=+78.998691632" watchObservedRunningTime="2026-07-24 22:36:55.792089607 +0000 UTC m=+78.999328418" Jul 24 22:36:55.808422 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.808335 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-qrkgm" podStartSLOduration=48.17918539 podStartE2EDuration="57.808323939s" totalImagesPullingTime="9.629138549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:55.807756613 +0000 UTC m=+79.014995421" watchObservedRunningTime="2026-07-24 22:36:55.808323939 +0000 UTC m=+79.015562871" Jul 24 22:36:55.827512 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:36:55.827469 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mfq5d" podStartSLOduration=67.620419277 podStartE2EDuration="1m17.827456457s" totalImagesPullingTime="10.20703718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:36:55.826995732 +0000 UTC m=+79.034234541" watchObservedRunningTime="2026-07-24 22:36:55.827456457 +0000 UTC m=+79.034695263" Jul 24 22:37:00.776755 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:00.776713 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bmgbj" event={"ID":"788fe0e1-ae93-41a9-a3be-2890ca680fd0","Type":"ContainerStarted","Data":"6894649adf693a4b6f8502fe92a812d511d46eabc7319686e3b44ba46e0c770a"} Jul 24 22:37:00.798732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:00.798676 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bmgbj" podStartSLOduration=17.720039016 podStartE2EDuration="37.798654942s" totalImagesPullingTime="20.078615926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:36:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:00.796494639 +0000 UTC m=+84.003733468" watchObservedRunningTime="2026-07-24 22:37:00.798654942 +0000 UTC m=+84.005893756" Jul 24 22:37:04.823792 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:04.823753 2559 patch_prober.go:28] interesting pod/image-registry-79c8d7d4cd-df4fz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 22:37:04.824403 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:04.823817 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 22:37:05.760365 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:05.760332 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mdjzv" Jul 24 22:37:06.693966 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:06.693940 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:37:08.756489 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.756451 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-prhdt"] Jul 24 22:37:08.783987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.783942 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.789881 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.789852 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zzs55\"" Jul 24 22:37:08.790020 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.789939 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 22:37:08.790084 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.790061 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 22:37:08.790148 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.790095 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 22:37:08.791279 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.790283 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 22:37:08.818755 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.818682 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-cwnrf" event={"ID":"435b83b2-108b-4edb-becb-c7c3702db1d2","Type":"ContainerStarted","Data":"a96e87482d081a1c8322fa00e0359969beb9013b083d9155a3491fe9563d834b"} Jul 24 22:37:08.822520 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822495 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-metrics-client-ca\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-textfile\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-root\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822603 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-tls\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822639 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-accelerators-collector-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822663 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv888\" (UniqueName: \"kubernetes.io/projected/f2b2a315-0a55-4017-8137-416a517c4463-kube-api-access-zv888\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822737 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-sys\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.822858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.822767 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-wtmp\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.840118 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.840058 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-cwnrf" podStartSLOduration=2.048369275 podStartE2EDuration="22.840040362s" totalImagesPullingTime="20.791671087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:08.838032041 +0000 UTC m=+92.045271056" watchObservedRunningTime="2026-07-24 22:37:08.840040362 +0000 UTC m=+92.047279171" Jul 24 22:37:08.923771 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.923975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-sys\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.923975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923807 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-wtmp\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.923975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923887 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-metrics-client-ca\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.923975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923946 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-textfile\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.923975 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-root\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.924214 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.923988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-tls\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.924214 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.924028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-accelerators-collector-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.924214 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.924054 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv888\" (UniqueName: \"kubernetes.io/projected/f2b2a315-0a55-4017-8137-416a517c4463-kube-api-access-zv888\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.925487 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.925465 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-textfile\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.925610 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.925535 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-sys\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.925673 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.925629 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-root\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.925673 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.925643 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-wtmp\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.926068 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.926045 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-metrics-client-ca\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.926558 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.926537 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-accelerators-collector-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.928181 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.928137 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.934253 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.934192 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv888\" (UniqueName: \"kubernetes.io/projected/f2b2a315-0a55-4017-8137-416a517c4463-kube-api-access-zv888\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:08.935815 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:08.935792 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2b2a315-0a55-4017-8137-416a517c4463-node-exporter-tls\") pod \"node-exporter-prhdt\" (UID: \"f2b2a315-0a55-4017-8137-416a517c4463\") " pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:09.097916 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:09.097874 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-prhdt" Jul 24 22:37:09.138069 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:37:09.138027 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b2a315_0a55_4017_8137_416a517c4463.slice/crio-8058df81b562a8b32e5271db28da83e04120c4bb2d684d2b4ae279b50d084a48 WatchSource:0}: Error finding container 8058df81b562a8b32e5271db28da83e04120c4bb2d684d2b4ae279b50d084a48: Status 404 returned error can't find the container with id 8058df81b562a8b32e5271db28da83e04120c4bb2d684d2b4ae279b50d084a48 Jul 24 22:37:09.851499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:09.851252 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prhdt" event={"ID":"f2b2a315-0a55-4017-8137-416a517c4463","Type":"ContainerStarted","Data":"8058df81b562a8b32e5271db28da83e04120c4bb2d684d2b4ae279b50d084a48"} Jul 24 22:37:09.851499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:09.851321 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:37:09.857539 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:09.857281 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-cwnrf" Jul 24 22:37:11.065181 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:11.065153 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:37:11.859746 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:11.859709 2559 generic.go:356] "Generic (PLEG): container finished" podID="f2b2a315-0a55-4017-8137-416a517c4463" containerID="1618bba4e9d89b7fd9eea8489b9d0d8cada4d7d040f8986a4ba37d1389cba2f0" exitCode=0 Jul 24 22:37:11.859936 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:11.859799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prhdt" event={"ID":"f2b2a315-0a55-4017-8137-416a517c4463","Type":"ContainerDied","Data":"1618bba4e9d89b7fd9eea8489b9d0d8cada4d7d040f8986a4ba37d1389cba2f0"} Jul 24 22:37:12.867172 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:12.867134 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prhdt" event={"ID":"f2b2a315-0a55-4017-8137-416a517c4463","Type":"ContainerStarted","Data":"611ed1f80e4564ae40828a663992d613cafb43b67ac3b1d0349d081f73152fe1"} Jul 24 22:37:12.867641 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:12.867197 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prhdt" event={"ID":"f2b2a315-0a55-4017-8137-416a517c4463","Type":"ContainerStarted","Data":"5fd093430e0d8dbd117a92c1a8f44f92b5ada953bfc230d2d09757c0fe8d65d9"} Jul 24 22:37:12.889458 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:12.889388 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-prhdt" podStartSLOduration=2.93903968 podStartE2EDuration="4.889368505s" totalImagesPullingTime="1.812093347s" totalInitContainerRuntime="138.235478ms" isStatefulPod=true podCreationTimestamp="2026-07-24 22:37:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:12.888308888 +0000 UTC m=+96.095547709" watchObservedRunningTime="2026-07-24 22:37:12.889368505 +0000 UTC m=+96.096607342" Jul 24 22:37:15.681740 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:15.681703 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vpb69" Jul 24 22:37:28.940838 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.940806 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" event={"ID":"b6cc603d-dc69-4a7a-b97d-0d89b4414ff0","Type":"ContainerStarted","Data":"11f6157f2fd2a74ed9c88fd9ecdb2f9f5802c0a0da19fbc235423fe6cdbc84c2"} Jul 24 22:37:28.942315 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.942290 2559 generic.go:356] "Generic (PLEG): container finished" podID="40f6c541-38fd-4e28-8d80-80f2fa2c7cb3" containerID="57a14940df702e6f9fcefcd102d79afaca96fff4cbdaf2702adfec7973ec1939" exitCode=0 Jul 24 22:37:28.942416 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.942358 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qdl87" event={"ID":"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3","Type":"ContainerDied","Data":"57a14940df702e6f9fcefcd102d79afaca96fff4cbdaf2702adfec7973ec1939"} Jul 24 22:37:28.942744 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.942724 2559 scope.go:117] "RemoveContainer" containerID="57a14940df702e6f9fcefcd102d79afaca96fff4cbdaf2702adfec7973ec1939" Jul 24 22:37:28.943949 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.943927 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" event={"ID":"e8c079b5-0c68-4a74-a210-4a91ad677ea9","Type":"ContainerStarted","Data":"bc97bd0fb9d9cd8dc8de05caa97c01faf57c1b573547e2022f541c7b907a966f"} Jul 24 22:37:28.945600 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.945578 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" event={"ID":"1441bb85-82b6-482c-ab37-713eda4c2a79","Type":"ContainerStarted","Data":"e52bcf1c7d87d214518a6f6d1618c75487826b1ee958c7f1404e6a25e2674d51"} Jul 24 22:37:28.945682 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.945608 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" event={"ID":"1441bb85-82b6-482c-ab37-713eda4c2a79","Type":"ContainerStarted","Data":"d05fd790e5d3627ce4c6ca43ecaa0550c71c5ba4ea7c85b7d8302b5b4f25a65d"} Jul 24 22:37:28.946083 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.946020 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:37:28.947824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.947802 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" Jul 24 22:37:28.980558 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.980407 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69cfdcdb48-vnw7t" podStartSLOduration=1.403521706 podStartE2EDuration="42.980390735s" totalImagesPullingTime="41.576869029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:28.979757954 +0000 UTC m=+112.186996763" watchObservedRunningTime="2026-07-24 22:37:28.980390735 +0000 UTC m=+112.187629542" Jul 24 22:37:28.998685 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:28.998631 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5bd56b6d5f-zxvfx" podStartSLOduration=1.465967238 podStartE2EDuration="42.998609556s" totalImagesPullingTime="41.532642318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:28.997504455 +0000 UTC m=+112.204743265" watchObservedRunningTime="2026-07-24 22:37:28.998609556 +0000 UTC m=+112.205848369" Jul 24 22:37:29.952324 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:29.952275 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" event={"ID":"1441bb85-82b6-482c-ab37-713eda4c2a79","Type":"ContainerStarted","Data":"e1e04636b650197f02ec14e84aa5a4ab647485d7293e95307057be2eca92535b"} Jul 24 22:37:29.954150 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:29.954124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qdl87" event={"ID":"40f6c541-38fd-4e28-8d80-80f2fa2c7cb3","Type":"ContainerStarted","Data":"3c613c132ffe253cb2aa7e8bc46884fb39ae6c5a825e9c91eb5a99ce0b7c78a7"} Jul 24 22:37:29.975023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:29.974974 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-74f7b8f8fc-xhls2" podStartSLOduration=2.496635423 podStartE2EDuration="43.974958687s" totalImagesPullingTime="41.478323264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:37:29.973135172 +0000 UTC m=+113.180374008" watchObservedRunningTime="2026-07-24 22:37:29.974958687 +0000 UTC m=+113.182197494" Jul 24 22:37:36.088963 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.088922 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerName="registry" containerID="cri-o://a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906" gracePeriod=30 Jul 24 22:37:36.354010 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.353949 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:37:36.473732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473696 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473732 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473736 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473770 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473807 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473830 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473853 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.473940 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473875 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlx5h\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.474164 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.473991 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration\") pod \"c76426fe-f152-460b-b873-3d2931d1e7ac\" (UID: \"c76426fe-f152-460b-b873-3d2931d1e7ac\") " Jul 24 22:37:36.474331 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.474301 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:37:36.474517 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.474428 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:37:36.476294 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.476272 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:37:36.476410 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.476383 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:37:36.476802 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.476772 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h" (OuterVolumeSpecName: "kube-api-access-jlx5h") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "kube-api-access-jlx5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:37:36.476802 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.476776 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:37:36.476957 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.476922 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:37:36.482970 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.482948 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c76426fe-f152-460b-b873-3d2931d1e7ac" (UID: "c76426fe-f152-460b-b873-3d2931d1e7ac"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:37:36.575541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575516 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-certificates\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575540 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-registry-tls\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575551 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-installation-pull-secrets\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575560 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c76426fe-f152-460b-b873-3d2931d1e7ac-trusted-ca\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575568 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c76426fe-f152-460b-b873-3d2931d1e7ac-ca-trust-extracted\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575576 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-bound-sa-token\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575583 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlx5h\" (UniqueName: \"kubernetes.io/projected/c76426fe-f152-460b-b873-3d2931d1e7ac-kube-api-access-jlx5h\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.575718 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.575595 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c76426fe-f152-460b-b873-3d2931d1e7ac-image-registry-private-configuration\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:37:36.982762 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.982732 2559 generic.go:356] "Generic (PLEG): container finished" podID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerID="a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906" exitCode=0 Jul 24 22:37:36.983003 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.982780 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" Jul 24 22:37:36.983003 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.982820 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" event={"ID":"c76426fe-f152-460b-b873-3d2931d1e7ac","Type":"ContainerDied","Data":"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906"} Jul 24 22:37:36.983003 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.982871 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79c8d7d4cd-df4fz" event={"ID":"c76426fe-f152-460b-b873-3d2931d1e7ac","Type":"ContainerDied","Data":"caea038cc30648452054c447f807dd9f6f438872b60b9e60f03fcac90b17b7a1"} Jul 24 22:37:36.983003 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.982904 2559 scope.go:117] "RemoveContainer" containerID="a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906" Jul 24 22:37:36.991678 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.991660 2559 scope.go:117] "RemoveContainer" containerID="a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906" Jul 24 22:37:36.991939 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:37:36.991910 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906\": container with ID starting with a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906 not found: ID does not exist" containerID="a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906" Jul 24 22:37:36.991989 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:36.991950 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906"} err="failed to get container status \"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906\": rpc error: code = NotFound desc = could not find container \"a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906\": container with ID starting with a2d138d760d51bb2e2dc40af702b21a8af4eb4be9359cf382d2102f3184e7906 not found: ID does not exist" Jul 24 22:37:37.004006 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:37.003985 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:37:37.009683 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:37.009663 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79c8d7d4cd-df4fz"] Jul 24 22:37:37.218441 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:37.218412 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" path="/var/lib/kubelet/pods/c76426fe-f152-460b-b873-3d2931d1e7ac/volumes" Jul 24 22:37:53.044650 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:53.044620 2559 generic.go:356] "Generic (PLEG): container finished" podID="69dea1e1-26cb-408d-b7db-23f1aeed1250" containerID="0d3eacb55cd3eec2c7b44fb8bbcd3f55d56c4a062ea24be26f5ac6e678cd2455" exitCode=0 Jul 24 22:37:53.045050 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:53.044686 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" event={"ID":"69dea1e1-26cb-408d-b7db-23f1aeed1250","Type":"ContainerDied","Data":"0d3eacb55cd3eec2c7b44fb8bbcd3f55d56c4a062ea24be26f5ac6e678cd2455"} Jul 24 22:37:53.045050 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:53.044979 2559 scope.go:117] "RemoveContainer" containerID="0d3eacb55cd3eec2c7b44fb8bbcd3f55d56c4a062ea24be26f5ac6e678cd2455" Jul 24 22:37:54.051305 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:54.051268 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-npd46" event={"ID":"69dea1e1-26cb-408d-b7db-23f1aeed1250","Type":"ContainerStarted","Data":"4920e66317fc51f11522d598a167fee0e390f9f4b4720e253cf68ad4fc346d11"} Jul 24 22:37:58.067605 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:58.067568 2559 generic.go:356] "Generic (PLEG): container finished" podID="20729795-56c7-49cc-934c-d9df6f309413" containerID="7bbc464562121adc0540b35a0f8775d786c5cde3da7f50daddafe1c6f53247c6" exitCode=0 Jul 24 22:37:58.068053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:58.067641 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" event={"ID":"20729795-56c7-49cc-934c-d9df6f309413","Type":"ContainerDied","Data":"7bbc464562121adc0540b35a0f8775d786c5cde3da7f50daddafe1c6f53247c6"} Jul 24 22:37:58.068053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:58.067980 2559 scope.go:117] "RemoveContainer" containerID="7bbc464562121adc0540b35a0f8775d786c5cde3da7f50daddafe1c6f53247c6" Jul 24 22:37:59.073839 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:37:59.073805 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qtxj2" event={"ID":"20729795-56c7-49cc-934c-d9df6f309413","Type":"ContainerStarted","Data":"0a2ceb83921522dae247fcf61d3a6397131d27855483493b9ddc29eb673a9f4e"} Jul 24 22:40:37.118557 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:37.118527 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:40:37.121483 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:37.121459 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:40:37.122847 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:37.122820 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:40:37.126245 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:37.126209 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:40:37.128603 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:37.128583 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 22:40:40.537671 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.537635 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz"] Jul 24 22:40:40.538928 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.537927 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerName="registry" Jul 24 22:40:40.538928 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.537938 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerName="registry" Jul 24 22:40:40.538928 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.537990 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76426fe-f152-460b-b873-3d2931d1e7ac" containerName="registry" Jul 24 22:40:40.540559 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.540534 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.544351 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.544323 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 22:40:40.545683 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.545647 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 22:40:40.545795 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.545697 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fzfjr\"" Jul 24 22:40:40.545795 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.545739 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 22:40:40.546171 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.546141 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 22:40:40.558137 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.558112 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz"] Jul 24 22:40:40.643812 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.643773 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-manifests\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.643988 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.643858 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.643988 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.643884 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-tmp\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.643988 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.643967 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkj2r\" (UniqueName: \"kubernetes.io/projected/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-kube-api-access-vkj2r\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.644114 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.644011 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.744659 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.744626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkj2r\" (UniqueName: \"kubernetes.io/projected/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-kube-api-access-vkj2r\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.744659 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.744673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.744884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.744716 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-manifests\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.744884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.744771 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.744884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.744789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-tmp\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.745309 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.745286 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-manifests\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.747138 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.747114 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-tmp\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.747539 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.747519 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-webhook-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.747539 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.747527 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.753205 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.753183 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkj2r\" (UniqueName: \"kubernetes.io/projected/f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf-kube-api-access-vkj2r\") pod \"opendatahub-operator-controller-manager-665d7b955f-6z9qz\" (UID: \"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf\") " pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.852388 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.852356 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:40.992337 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.992304 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz"] Jul 24 22:40:40.996146 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:40:40.996115 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e2a12d_0cc7_48d9_b23b_2e0ca4560bdf.slice/crio-c5ca2925d3b9d563895643409f1a553205ba0a7edc74e53430764ce7177e76fe WatchSource:0}: Error finding container c5ca2925d3b9d563895643409f1a553205ba0a7edc74e53430764ce7177e76fe: Status 404 returned error can't find the container with id c5ca2925d3b9d563895643409f1a553205ba0a7edc74e53430764ce7177e76fe Jul 24 22:40:40.997885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:40.997866 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 22:40:41.686404 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:41.686365 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" event={"ID":"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf","Type":"ContainerStarted","Data":"c5ca2925d3b9d563895643409f1a553205ba0a7edc74e53430764ce7177e76fe"} Jul 24 22:40:44.701751 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:44.701718 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" event={"ID":"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf","Type":"ContainerStarted","Data":"cd9cf58102a7a45a5d9ef662a2578b8eafa96a6f01864fdf1523b9a0f572446f"} Jul 24 22:40:45.709888 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:45.709854 2559 generic.go:356] "Generic (PLEG): container finished" podID="f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf" containerID="cd9cf58102a7a45a5d9ef662a2578b8eafa96a6f01864fdf1523b9a0f572446f" exitCode=0 Jul 24 22:40:45.710278 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:45.709940 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" event={"ID":"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf","Type":"ContainerDied","Data":"cd9cf58102a7a45a5d9ef662a2578b8eafa96a6f01864fdf1523b9a0f572446f"} Jul 24 22:40:46.715840 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:46.715804 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" event={"ID":"f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf","Type":"ContainerStarted","Data":"0a39cc3c6e65d15d392cac9535aabc48a840b81066ec8a4b2e91a53d64008412"} Jul 24 22:40:46.716282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:46.715919 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:40:46.743394 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:46.743340 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" podStartSLOduration=2.33559407 podStartE2EDuration="6.743325709s" totalImagesPullingTime="3.374461255s" totalInitContainerRuntime="1.033270384s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:40:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:40:46.741532399 +0000 UTC m=+309.948771207" watchObservedRunningTime="2026-07-24 22:40:46.743325709 +0000 UTC m=+309.950564517" Jul 24 22:40:57.721881 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:40:57.721854 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-665d7b955f-6z9qz" Jul 24 22:41:00.247814 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.247784 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh"] Jul 24 22:41:00.251010 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.250993 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.253782 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.253759 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m7z6z\"" Jul 24 22:41:00.254076 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.254056 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 22:41:00.254793 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.254773 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 22:41:00.267913 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.267887 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh"] Jul 24 22:41:00.297004 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.296980 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704d40dc-469a-43bd-817c-4125ef46a881-tmp\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.297122 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.297032 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qmxs\" (UniqueName: \"kubernetes.io/projected/704d40dc-469a-43bd-817c-4125ef46a881-kube-api-access-8qmxs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.297122 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.297064 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.398086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.398061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qmxs\" (UniqueName: \"kubernetes.io/projected/704d40dc-469a-43bd-817c-4125ef46a881-kube-api-access-8qmxs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.398086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.398094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.398308 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.398148 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704d40dc-469a-43bd-817c-4125ef46a881-tmp\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.398365 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:00.398312 2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 24 22:41:00.398415 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:00.398381 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs podName:704d40dc-469a-43bd-817c-4125ef46a881 nodeName:}" failed. No retries permitted until 2026-07-24 22:41:00.898360027 +0000 UTC m=+324.105598825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs") pod "kube-auth-proxy-65f7697f5d-5whnh" (UID: "704d40dc-469a-43bd-817c-4125ef46a881") : secret "kube-auth-proxy-tls" not found Jul 24 22:41:00.400463 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.400437 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/704d40dc-469a-43bd-817c-4125ef46a881-tmp\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.410884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.410859 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qmxs\" (UniqueName: \"kubernetes.io/projected/704d40dc-469a-43bd-817c-4125ef46a881-kube-api-access-8qmxs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.465488 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.465462 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrtvn"] Jul 24 22:41:00.468804 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.468783 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.471831 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.471810 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 22:41:00.472860 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.472843 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tkmk6\"" Jul 24 22:41:00.486104 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.486082 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrtvn"] Jul 24 22:41:00.499381 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.499316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtd7\" (UniqueName: \"kubernetes.io/projected/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-kube-api-access-ngtd7\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.499475 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.499397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.600323 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.600293 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtd7\" (UniqueName: \"kubernetes.io/projected/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-kube-api-access-ngtd7\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.600456 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.600364 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.600497 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:00.600458 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 24 22:41:00.600531 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:00.600511 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert podName:2b1a43b1-463c-468d-b62c-a1bdfdd93aef nodeName:}" failed. No retries permitted until 2026-07-24 22:41:01.100494694 +0000 UTC m=+324.307733483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert") pod "odh-model-controller-7998c67dc4-hrtvn" (UID: "2b1a43b1-463c-468d-b62c-a1bdfdd93aef") : secret "odh-model-controller-webhook-cert" not found Jul 24 22:41:00.619140 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.619105 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtd7\" (UniqueName: \"kubernetes.io/projected/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-kube-api-access-ngtd7\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:00.903219 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.903185 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:00.905751 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:00.905718 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/704d40dc-469a-43bd-817c-4125ef46a881-tls-certs\") pod \"kube-auth-proxy-65f7697f5d-5whnh\" (UID: \"704d40dc-469a-43bd-817c-4125ef46a881\") " pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:01.105339 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.105300 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:01.107771 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.107751 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b1a43b1-463c-468d-b62c-a1bdfdd93aef-cert\") pod \"odh-model-controller-7998c67dc4-hrtvn\" (UID: \"2b1a43b1-463c-468d-b62c-a1bdfdd93aef\") " pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:01.160765 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.160682 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" Jul 24 22:41:01.313621 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.313595 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh"] Jul 24 22:41:01.315393 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:41:01.315354 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704d40dc_469a_43bd_817c_4125ef46a881.slice/crio-8c252a29102e2d2e2d28760e3da9904f78a150aa3021136ffe484cb37ad95d54 WatchSource:0}: Error finding container 8c252a29102e2d2e2d28760e3da9904f78a150aa3021136ffe484cb37ad95d54: Status 404 returned error can't find the container with id 8c252a29102e2d2e2d28760e3da9904f78a150aa3021136ffe484cb37ad95d54 Jul 24 22:41:01.380549 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.380522 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:01.533333 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:41:01.533302 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1a43b1_463c_468d_b62c_a1bdfdd93aef.slice/crio-8186a54a7e5bb16913f1d07ab57d7ad5968a8cc5453a68b80f0c7a1b284f1c8d WatchSource:0}: Error finding container 8186a54a7e5bb16913f1d07ab57d7ad5968a8cc5453a68b80f0c7a1b284f1c8d: Status 404 returned error can't find the container with id 8186a54a7e5bb16913f1d07ab57d7ad5968a8cc5453a68b80f0c7a1b284f1c8d Jul 24 22:41:01.534735 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.534715 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-hrtvn"] Jul 24 22:41:01.632315 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.632283 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z8ttg"] Jul 24 22:41:01.637163 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.637146 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.641741 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.641716 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 22:41:01.641909 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.641893 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-pzqqt\"" Jul 24 22:41:01.651132 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.651109 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z8ttg"] Jul 24 22:41:01.709919 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.709850 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.709919 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.709903 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7jj\" (UniqueName: \"kubernetes.io/projected/db6acc5f-f292-4d42-ab69-3a7719de8da5-kube-api-access-rq7jj\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.786067 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.786036 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" event={"ID":"704d40dc-469a-43bd-817c-4125ef46a881","Type":"ContainerStarted","Data":"8c252a29102e2d2e2d28760e3da9904f78a150aa3021136ffe484cb37ad95d54"} Jul 24 22:41:01.787196 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.787166 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" event={"ID":"2b1a43b1-463c-468d-b62c-a1bdfdd93aef","Type":"ContainerStarted","Data":"8186a54a7e5bb16913f1d07ab57d7ad5968a8cc5453a68b80f0c7a1b284f1c8d"} Jul 24 22:41:01.810663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.810640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.810750 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.810692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7jj\" (UniqueName: \"kubernetes.io/projected/db6acc5f-f292-4d42-ab69-3a7719de8da5-kube-api-access-rq7jj\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.810824 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:01.810806 2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 24 22:41:01.810889 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:01.810879 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs podName:db6acc5f-f292-4d42-ab69-3a7719de8da5 nodeName:}" failed. No retries permitted until 2026-07-24 22:41:02.310858582 +0000 UTC m=+325.518097372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs") pod "model-serving-api-54d89bd5fc-z8ttg" (UID: "db6acc5f-f292-4d42-ab69-3a7719de8da5") : secret "model-serving-api-tls" not found Jul 24 22:41:01.822741 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.822720 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7jj\" (UniqueName: \"kubernetes.io/projected/db6acc5f-f292-4d42-ab69-3a7719de8da5-kube-api-access-rq7jj\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:01.903117 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.899689 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:01.904517 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.904491 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:01.907376 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.907350 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 22:41:01.907655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.907636 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 22:41:01.907714 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.907663 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 22:41:01.907714 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.907677 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wdnk6\"" Jul 24 22:41:01.908771 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.908753 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 22:41:01.915065 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:01.915043 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:02.012394 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012394 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx58b\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012581 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012442 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012581 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012466 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012581 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012581 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012564 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.012785 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.012595 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113661 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113630 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113672 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113743 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113771 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx58b\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113822 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.113853 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113845 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.114120 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.113866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.115142 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.115112 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.116549 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.116530 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.116675 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.116653 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.116759 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.116735 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.117119 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.117096 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.125582 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.125557 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx58b\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.125894 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.125852 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-j4thl\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.214904 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.214777 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:02.316201 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.316162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:02.323808 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.323748 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db6acc5f-f292-4d42-ab69-3a7719de8da5-tls-certs\") pod \"model-serving-api-54d89bd5fc-z8ttg\" (UID: \"db6acc5f-f292-4d42-ab69-3a7719de8da5\") " pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:02.426478 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.426196 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:02.431030 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:41:02.430993 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945154e6_13da_441c_b0d4_a980ff5ef636.slice/crio-6e1d0d6911ab379b68c3d05c620270423e5940d90c8dd0ae9049ee89e82cbd4c WatchSource:0}: Error finding container 6e1d0d6911ab379b68c3d05c620270423e5940d90c8dd0ae9049ee89e82cbd4c: Status 404 returned error can't find the container with id 6e1d0d6911ab379b68c3d05c620270423e5940d90c8dd0ae9049ee89e82cbd4c Jul 24 22:41:02.550706 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.550631 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:02.748051 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.747962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z8ttg"] Jul 24 22:41:02.753405 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:41:02.753365 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb6acc5f_f292_4d42_ab69_3a7719de8da5.slice/crio-a3cf422f8d2c8dfdb999f0bacd00761830f04329d9a442968ca812ddb529ca64 WatchSource:0}: Error finding container a3cf422f8d2c8dfdb999f0bacd00761830f04329d9a442968ca812ddb529ca64: Status 404 returned error can't find the container with id a3cf422f8d2c8dfdb999f0bacd00761830f04329d9a442968ca812ddb529ca64 Jul 24 22:41:02.796574 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.796539 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" event={"ID":"945154e6-13da-441c-b0d4-a980ff5ef636","Type":"ContainerStarted","Data":"6e1d0d6911ab379b68c3d05c620270423e5940d90c8dd0ae9049ee89e82cbd4c"} Jul 24 22:41:02.798537 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:02.798509 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" event={"ID":"db6acc5f-f292-4d42-ab69-3a7719de8da5","Type":"ContainerStarted","Data":"a3cf422f8d2c8dfdb999f0bacd00761830f04329d9a442968ca812ddb529ca64"} Jul 24 22:41:06.825993 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.825952 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" event={"ID":"db6acc5f-f292-4d42-ab69-3a7719de8da5","Type":"ContainerStarted","Data":"77542f583eb905ab2459b20440bdb5faa905af8c48af461a55fc2a9c41a35e43"} Jul 24 22:41:06.826560 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.826115 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:06.827672 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.827647 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" event={"ID":"704d40dc-469a-43bd-817c-4125ef46a881","Type":"ContainerStarted","Data":"c4bd3260c5893766bef412d831f2714e92db311429bc492e2d41eab6d38ad015"} Jul 24 22:41:06.829459 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.829429 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" event={"ID":"2b1a43b1-463c-468d-b62c-a1bdfdd93aef","Type":"ContainerStarted","Data":"05900acb595d6bdc2ddc80e7455e3f0780cc9359c6f8f335e24bf29e7a1304a3"} Jul 24 22:41:06.829905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.829853 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:06.846449 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.846403 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" podStartSLOduration=2.643941924 podStartE2EDuration="5.846371923s" totalImagesPullingTime="3.202429999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:41:06.844382702 +0000 UTC m=+330.051621511" watchObservedRunningTime="2026-07-24 22:41:06.846371923 +0000 UTC m=+330.053610735" Jul 24 22:41:06.861427 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.861379 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" podStartSLOduration=2.422155562 podStartE2EDuration="6.861364346s" totalImagesPullingTime="4.439208784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:41:06.86041962 +0000 UTC m=+330.067658431" watchObservedRunningTime="2026-07-24 22:41:06.861364346 +0000 UTC m=+330.068603152" Jul 24 22:41:06.878088 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:06.878029 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65f7697f5d-5whnh" podStartSLOduration=2.236254615 podStartE2EDuration="6.878012336s" totalImagesPullingTime="4.641757721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:41:06.876301081 +0000 UTC m=+330.083539909" watchObservedRunningTime="2026-07-24 22:41:06.878012336 +0000 UTC m=+330.085251144" Jul 24 22:41:07.378141 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:07.378095 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 22:41:07.378282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:07.378183 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 22:41:07.835728 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:07.835625 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" event={"ID":"945154e6-13da-441c-b0d4-a980ff5ef636","Type":"ContainerStarted","Data":"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919"} Jul 24 22:41:07.836144 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:07.835839 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:07.881355 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:07.881305 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" podStartSLOduration=1.9378147650000002 podStartE2EDuration="6.881291894s" totalImagesPullingTime="4.943477129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:41:07.878849576 +0000 UTC m=+331.086088384" watchObservedRunningTime="2026-07-24 22:41:07.881291894 +0000 UTC m=+331.088530702" Jul 24 22:41:08.843558 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:08.843518 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:11.970125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:11.970094 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q82fl"] Jul 24 22:41:11.973855 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:11.973831 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:11.976639 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:11.976615 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-m2nf4\"" Jul 24 22:41:11.976881 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:11.976866 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 24 22:41:11.995420 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:11.995384 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q82fl"] Jul 24 22:41:12.107426 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.107386 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plwrr\" (UniqueName: \"kubernetes.io/projected/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-kube-api-access-plwrr\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.107611 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.107449 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-cert\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.208001 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.207948 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plwrr\" (UniqueName: \"kubernetes.io/projected/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-kube-api-access-plwrr\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.208175 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.208040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-cert\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.210572 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.210548 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-cert\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.224071 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.224009 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plwrr\" (UniqueName: \"kubernetes.io/projected/23f06a1d-fea8-4cc8-8cd3-97c607db5d78-kube-api-access-plwrr\") pod \"kserve-controller-manager-d6f4b65b8-q82fl\" (UID: \"23f06a1d-fea8-4cc8-8cd3-97c607db5d78\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.290410 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.290370 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:12.452521 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.452483 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q82fl"] Jul 24 22:41:12.459921 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:41:12.459887 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f06a1d_fea8_4cc8_8cd3_97c607db5d78.slice/crio-77c9dfedcd64e033f594e498b2c89378cc2afaa5f75ddee56a2253570cd9a4e5 WatchSource:0}: Error finding container 77c9dfedcd64e033f594e498b2c89378cc2afaa5f75ddee56a2253570cd9a4e5: Status 404 returned error can't find the container with id 77c9dfedcd64e033f594e498b2c89378cc2afaa5f75ddee56a2253570cd9a4e5 Jul 24 22:41:12.859266 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:12.859215 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" event={"ID":"23f06a1d-fea8-4cc8-8cd3-97c607db5d78","Type":"ContainerStarted","Data":"77c9dfedcd64e033f594e498b2c89378cc2afaa5f75ddee56a2253570cd9a4e5"} Jul 24 22:41:13.945267 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:13.945200 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:13.945773 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:13.945469 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" podUID="945154e6-13da-441c-b0d4-a980ff5ef636" containerName="discovery" containerID="cri-o://a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919" gracePeriod=30 Jul 24 22:41:14.745953 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.745927 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:14.833254 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833200 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833269 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833298 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx58b\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833401 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833551 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833462 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833551 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833499 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.833551 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.833546 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts\") pod \"945154e6-13da-441c-b0d4-a980ff5ef636\" (UID: \"945154e6-13da-441c-b0d4-a980ff5ef636\") " Jul 24 22:41:14.838707 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.838632 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:41:14.840701 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.840671 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token" (OuterVolumeSpecName: "istio-token") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:41:14.840811 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.840714 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts" (OuterVolumeSpecName: "cacerts") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:41:14.840920 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.840869 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:41:14.840920 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.840906 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:41:14.841249 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.841205 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs" (OuterVolumeSpecName: "local-certs") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:41:14.841328 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.841309 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b" (OuterVolumeSpecName: "kube-api-access-bx58b") pod "945154e6-13da-441c-b0d4-a980ff5ef636" (UID: "945154e6-13da-441c-b0d4-a980ff5ef636"). InnerVolumeSpecName "kube-api-access-bx58b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:41:14.871700 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.871671 2559 generic.go:356] "Generic (PLEG): container finished" podID="945154e6-13da-441c-b0d4-a980ff5ef636" containerID="a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919" exitCode=0 Jul 24 22:41:14.871885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.871723 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" Jul 24 22:41:14.871885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.871753 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" event={"ID":"945154e6-13da-441c-b0d4-a980ff5ef636","Type":"ContainerDied","Data":"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919"} Jul 24 22:41:14.871885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.871810 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl" event={"ID":"945154e6-13da-441c-b0d4-a980ff5ef636","Type":"ContainerDied","Data":"6e1d0d6911ab379b68c3d05c620270423e5940d90c8dd0ae9049ee89e82cbd4c"} Jul 24 22:41:14.871885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.871837 2559 scope.go:117] "RemoveContainer" containerID="a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919" Jul 24 22:41:14.873738 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.873710 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" event={"ID":"23f06a1d-fea8-4cc8-8cd3-97c607db5d78","Type":"ContainerStarted","Data":"dafcd25f7b449c29391831a9991458df708d26afaf149999259ffc272ba91772"} Jul 24 22:41:14.873933 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.873914 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:41:14.881591 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.881574 2559 scope.go:117] "RemoveContainer" containerID="a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919" Jul 24 22:41:14.881875 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:41:14.881858 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919\": container with ID starting with a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919 not found: ID does not exist" containerID="a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919" Jul 24 22:41:14.881928 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.881884 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919"} err="failed to get container status \"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919\": rpc error: code = NotFound desc = could not find container \"a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919\": container with ID starting with a788ea75a1a74892ce600171f2da49a058f9bffb1fa5a693e8c5795f587fb919 not found: ID does not exist" Jul 24 22:41:14.934880 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934842 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-istio-token\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.934880 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934870 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-ca-configmap\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.934880 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934883 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx58b\" (UniqueName: \"kubernetes.io/projected/945154e6-13da-441c-b0d4-a980ff5ef636-kube-api-access-bx58b\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.935129 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934896 2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/945154e6-13da-441c-b0d4-a980ff5ef636-local-certs\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.935129 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934910 2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-kubeconfig\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.935129 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934920 2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-istio-csr-dns-cert\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.935129 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.934930 2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/945154e6-13da-441c-b0d4-a980ff5ef636-cacerts\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:41:14.942253 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.942187 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" podStartSLOduration=1.765203481 podStartE2EDuration="3.942169504s" totalImagesPullingTime="2.176966023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:41:14.910822176 +0000 UTC m=+338.118060980" watchObservedRunningTime="2026-07-24 22:41:14.942169504 +0000 UTC m=+338.149408312" Jul 24 22:41:14.943640 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.943622 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:14.950795 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:14.950759 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-j4thl"] Jul 24 22:41:15.213434 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:15.213343 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="945154e6-13da-441c-b0d4-a980ff5ef636" path="/var/lib/kubelet/pods/945154e6-13da-441c-b0d4-a980ff5ef636/volumes" Jul 24 22:41:17.837884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:17.837857 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-hrtvn" Jul 24 22:41:17.840845 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:17.840819 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-z8ttg" Jul 24 22:41:45.884871 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:41:45.884837 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q82fl" Jul 24 22:42:01.423810 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.423779 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:01.424327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.424108 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="945154e6-13da-441c-b0d4-a980ff5ef636" containerName="discovery" Jul 24 22:42:01.424327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.424119 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="945154e6-13da-441c-b0d4-a980ff5ef636" containerName="discovery" Jul 24 22:42:01.424327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.424176 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="945154e6-13da-441c-b0d4-a980ff5ef636" containerName="discovery" Jul 24 22:42:01.427033 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.427018 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:01.430402 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.430380 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 22:42:01.431667 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.431650 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2tn4q\"" Jul 24 22:42:01.431738 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.431707 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 22:42:01.444151 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.444126 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:01.490938 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.490911 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84n5v\" (UniqueName: \"kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v\") pod \"kuadrant-operator-catalog-tk2mz\" (UID: \"42b3c4f7-496c-4d8a-9c64-93dda10c41a8\") " pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:01.591559 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.591528 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84n5v\" (UniqueName: \"kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v\") pod \"kuadrant-operator-catalog-tk2mz\" (UID: \"42b3c4f7-496c-4d8a-9c64-93dda10c41a8\") " pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:01.602257 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.602214 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84n5v\" (UniqueName: \"kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v\") pod \"kuadrant-operator-catalog-tk2mz\" (UID: \"42b3c4f7-496c-4d8a-9c64-93dda10c41a8\") " pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:01.737023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.736956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:01.749680 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.749654 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:01.884441 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.884388 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:01.887168 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:01.887144 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b3c4f7_496c_4d8a_9c64_93dda10c41a8.slice/crio-2d0277df710aae849154c412ae671d6f607d31f51fbc993d090b8bf7bfdc4084 WatchSource:0}: Error finding container 2d0277df710aae849154c412ae671d6f607d31f51fbc993d090b8bf7bfdc4084: Status 404 returned error can't find the container with id 2d0277df710aae849154c412ae671d6f607d31f51fbc993d090b8bf7bfdc4084 Jul 24 22:42:01.977033 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.977004 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-47cw7"] Jul 24 22:42:01.981411 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:01.981390 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:02.002660 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.002642 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-47cw7"] Jul 24 22:42:02.082031 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.082004 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" event={"ID":"42b3c4f7-496c-4d8a-9c64-93dda10c41a8","Type":"ContainerStarted","Data":"2d0277df710aae849154c412ae671d6f607d31f51fbc993d090b8bf7bfdc4084"} Jul 24 22:42:02.096541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.096519 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm5rt\" (UniqueName: \"kubernetes.io/projected/3dc03afe-c0a4-410f-a5f8-8697816c3b49-kube-api-access-cm5rt\") pod \"kuadrant-operator-catalog-47cw7\" (UID: \"3dc03afe-c0a4-410f-a5f8-8697816c3b49\") " pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:02.196872 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.196848 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm5rt\" (UniqueName: \"kubernetes.io/projected/3dc03afe-c0a4-410f-a5f8-8697816c3b49-kube-api-access-cm5rt\") pod \"kuadrant-operator-catalog-47cw7\" (UID: \"3dc03afe-c0a4-410f-a5f8-8697816c3b49\") " pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:02.207883 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.207856 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm5rt\" (UniqueName: \"kubernetes.io/projected/3dc03afe-c0a4-410f-a5f8-8697816c3b49-kube-api-access-cm5rt\") pod \"kuadrant-operator-catalog-47cw7\" (UID: \"3dc03afe-c0a4-410f-a5f8-8697816c3b49\") " pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:02.291343 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.291284 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:02.428979 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:02.428952 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-47cw7"] Jul 24 22:42:02.434360 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:02.434337 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc03afe_c0a4_410f_a5f8_8697816c3b49.slice/crio-2b45207c0d21a5475bced89a62d3a591bc7d677a933ba4ac5743019ea0430532 WatchSource:0}: Error finding container 2b45207c0d21a5475bced89a62d3a591bc7d677a933ba4ac5743019ea0430532: Status 404 returned error can't find the container with id 2b45207c0d21a5475bced89a62d3a591bc7d677a933ba4ac5743019ea0430532 Jul 24 22:42:03.094372 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:03.094336 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" event={"ID":"3dc03afe-c0a4-410f-a5f8-8697816c3b49","Type":"ContainerStarted","Data":"2b45207c0d21a5475bced89a62d3a591bc7d677a933ba4ac5743019ea0430532"} Jul 24 22:42:05.105876 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.105842 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" event={"ID":"42b3c4f7-496c-4d8a-9c64-93dda10c41a8","Type":"ContainerStarted","Data":"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819"} Jul 24 22:42:05.106379 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.105876 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" podUID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" containerName="registry-server" containerID="cri-o://0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819" gracePeriod=2 Jul 24 22:42:05.107310 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.107284 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" event={"ID":"3dc03afe-c0a4-410f-a5f8-8697816c3b49","Type":"ContainerStarted","Data":"6b2afa70a6fd03261915dbb75687dded713d1a60c36a62d6c4b8bb9f65776434"} Jul 24 22:42:05.126125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.126083 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" podStartSLOduration=1.705344572 podStartE2EDuration="4.12606915s" totalImagesPullingTime="2.420724578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:05.124477146 +0000 UTC m=+388.331715955" watchObservedRunningTime="2026-07-24 22:42:05.12606915 +0000 UTC m=+388.333307958" Jul 24 22:42:05.147002 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.146953 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" podStartSLOduration=2.270102128 podStartE2EDuration="4.146936796s" totalImagesPullingTime="1.876834668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:05.144492098 +0000 UTC m=+388.351730906" watchObservedRunningTime="2026-07-24 22:42:05.146936796 +0000 UTC m=+388.354175603" Jul 24 22:42:05.357394 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.357325 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:05.425723 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.425693 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84n5v\" (UniqueName: \"kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v\") pod \"42b3c4f7-496c-4d8a-9c64-93dda10c41a8\" (UID: \"42b3c4f7-496c-4d8a-9c64-93dda10c41a8\") " Jul 24 22:42:05.427895 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.427865 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v" (OuterVolumeSpecName: "kube-api-access-84n5v") pod "42b3c4f7-496c-4d8a-9c64-93dda10c41a8" (UID: "42b3c4f7-496c-4d8a-9c64-93dda10c41a8"). InnerVolumeSpecName "kube-api-access-84n5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:42:05.526583 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:05.526552 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84n5v\" (UniqueName: \"kubernetes.io/projected/42b3c4f7-496c-4d8a-9c64-93dda10c41a8-kube-api-access-84n5v\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:42:06.113663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.113626 2559 generic.go:356] "Generic (PLEG): container finished" podID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" containerID="0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819" exitCode=0 Jul 24 22:42:06.114079 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.113669 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" Jul 24 22:42:06.114079 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.113675 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" event={"ID":"42b3c4f7-496c-4d8a-9c64-93dda10c41a8","Type":"ContainerDied","Data":"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819"} Jul 24 22:42:06.114079 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.113732 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tk2mz" event={"ID":"42b3c4f7-496c-4d8a-9c64-93dda10c41a8","Type":"ContainerDied","Data":"2d0277df710aae849154c412ae671d6f607d31f51fbc993d090b8bf7bfdc4084"} Jul 24 22:42:06.114079 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.113754 2559 scope.go:117] "RemoveContainer" containerID="0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819" Jul 24 22:42:06.123159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.123139 2559 scope.go:117] "RemoveContainer" containerID="0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819" Jul 24 22:42:06.123438 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:06.123418 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819\": container with ID starting with 0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819 not found: ID does not exist" containerID="0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819" Jul 24 22:42:06.123499 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.123450 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819"} err="failed to get container status \"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819\": rpc error: code = NotFound desc = could not find container \"0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819\": container with ID starting with 0312a0c8aeca0662ba8ff936e4581f404093a9dc12213345ff56a40b822cd819 not found: ID does not exist" Jul 24 22:42:06.138519 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.138492 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:06.142677 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:06.142657 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tk2mz"] Jul 24 22:42:07.214072 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:07.214040 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" path="/var/lib/kubelet/pods/42b3c4f7-496c-4d8a-9c64-93dda10c41a8/volumes" Jul 24 22:42:12.292178 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:12.292144 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:12.292678 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:12.292202 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:12.314694 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:12.314667 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:13.167849 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:13.167824 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-47cw7" Jul 24 22:42:35.629425 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.629395 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4"] Jul 24 22:42:35.629807 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.629733 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" containerName="registry-server" Jul 24 22:42:35.629807 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.629743 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" containerName="registry-server" Jul 24 22:42:35.629877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.629809 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b3c4f7-496c-4d8a-9c64-93dda10c41a8" containerName="registry-server" Jul 24 22:42:35.637834 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.637813 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.640805 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.640784 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6554n\"" Jul 24 22:42:35.652153 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.652131 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4"] Jul 24 22:42:35.790402 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.790354 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.790572 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.790464 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmq5\" (UniqueName: \"kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.891392 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.891311 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.891392 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.891358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmq5\" (UniqueName: \"kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.891782 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.891758 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.902065 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.902040 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmq5\" (UniqueName: \"kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:35.947765 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:35.947746 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:36.087261 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:36.087216 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4"] Jul 24 22:42:36.091280 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:36.091259 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e02b98e_82b9_4277_82c4_cca7bdbae251.slice/crio-50d6abce0ba95511c0f0a9819d40519dc9bcc66ec8162bbcca4258f974a88cbc WatchSource:0}: Error finding container 50d6abce0ba95511c0f0a9819d40519dc9bcc66ec8162bbcca4258f974a88cbc: Status 404 returned error can't find the container with id 50d6abce0ba95511c0f0a9819d40519dc9bcc66ec8162bbcca4258f974a88cbc Jul 24 22:42:36.245248 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:36.245134 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" event={"ID":"6e02b98e-82b9-4277-82c4-cca7bdbae251","Type":"ContainerStarted","Data":"50d6abce0ba95511c0f0a9819d40519dc9bcc66ec8162bbcca4258f974a88cbc"} Jul 24 22:42:38.155862 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.155826 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-srzdk"] Jul 24 22:42:38.161098 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.161077 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:38.164994 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.164971 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-22dqx\"" Jul 24 22:42:38.175086 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.175066 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-srzdk"] Jul 24 22:42:38.210686 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.210659 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pz9l\" (UniqueName: \"kubernetes.io/projected/d0edc10f-ba97-4554-ba7a-87dd77e36654-kube-api-access-8pz9l\") pod \"authorino-operator-75cc8d5cb7-srzdk\" (UID: \"d0edc10f-ba97-4554-ba7a-87dd77e36654\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:38.311208 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.311176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pz9l\" (UniqueName: \"kubernetes.io/projected/d0edc10f-ba97-4554-ba7a-87dd77e36654-kube-api-access-8pz9l\") pod \"authorino-operator-75cc8d5cb7-srzdk\" (UID: \"d0edc10f-ba97-4554-ba7a-87dd77e36654\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:38.328537 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.328508 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pz9l\" (UniqueName: \"kubernetes.io/projected/d0edc10f-ba97-4554-ba7a-87dd77e36654-kube-api-access-8pz9l\") pod \"authorino-operator-75cc8d5cb7-srzdk\" (UID: \"d0edc10f-ba97-4554-ba7a-87dd77e36654\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:38.472410 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.472326 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:38.621423 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:38.621393 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-srzdk"] Jul 24 22:42:38.622954 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:38.622926 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0edc10f_ba97_4554_ba7a_87dd77e36654.slice/crio-b5702f7608ad8e3ff32b35dceb71d100f0ff498b1d96627ec3aaa54dda909217 WatchSource:0}: Error finding container b5702f7608ad8e3ff32b35dceb71d100f0ff498b1d96627ec3aaa54dda909217: Status 404 returned error can't find the container with id b5702f7608ad8e3ff32b35dceb71d100f0ff498b1d96627ec3aaa54dda909217 Jul 24 22:42:39.259440 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:39.259407 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" event={"ID":"d0edc10f-ba97-4554-ba7a-87dd77e36654","Type":"ContainerStarted","Data":"b5702f7608ad8e3ff32b35dceb71d100f0ff498b1d96627ec3aaa54dda909217"} Jul 24 22:42:40.949621 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:40.949582 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b"] Jul 24 22:42:40.953809 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:40.953785 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:40.956858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:40.956832 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dvd79\"" Jul 24 22:42:40.956858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:40.956848 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 22:42:40.976130 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:40.976101 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b"] Jul 24 22:42:41.030071 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:41.030033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfcqn\" (UniqueName: \"kubernetes.io/projected/cbd2bd20-e9ce-4d7e-897c-f23122eb370a-kube-api-access-xfcqn\") pod \"dns-operator-controller-manager-864dd78c4c-84c6b\" (UID: \"cbd2bd20-e9ce-4d7e-897c-f23122eb370a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:41.131581 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:41.131539 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfcqn\" (UniqueName: \"kubernetes.io/projected/cbd2bd20-e9ce-4d7e-897c-f23122eb370a-kube-api-access-xfcqn\") pod \"dns-operator-controller-manager-864dd78c4c-84c6b\" (UID: \"cbd2bd20-e9ce-4d7e-897c-f23122eb370a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:41.144431 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:41.144402 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfcqn\" (UniqueName: \"kubernetes.io/projected/cbd2bd20-e9ce-4d7e-897c-f23122eb370a-kube-api-access-xfcqn\") pod \"dns-operator-controller-manager-864dd78c4c-84c6b\" (UID: \"cbd2bd20-e9ce-4d7e-897c-f23122eb370a\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:41.266673 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:41.266572 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:42.411811 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.411788 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b"] Jul 24 22:42:42.414072 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:42.414040 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd2bd20_e9ce_4d7e_897c_f23122eb370a.slice/crio-1fb0045bd60f74caabed6e17f98026fff8af7563248510b15d9d0d20d6cb8807 WatchSource:0}: Error finding container 1fb0045bd60f74caabed6e17f98026fff8af7563248510b15d9d0d20d6cb8807: Status 404 returned error can't find the container with id 1fb0045bd60f74caabed6e17f98026fff8af7563248510b15d9d0d20d6cb8807 Jul 24 22:42:42.815986 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.815953 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx"] Jul 24 22:42:42.819593 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.819575 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:42.822545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.822525 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lf5bm\"" Jul 24 22:42:42.835633 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.835609 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx"] Jul 24 22:42:42.845871 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.845844 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqww7\" (UniqueName: \"kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7\") pod \"limitador-operator-controller-manager-859d7b78bd-6strx\" (UID: \"4bdb3c6e-7e04-45a1-9273-04618f0a39cb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:42.946832 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.946796 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqww7\" (UniqueName: \"kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7\") pod \"limitador-operator-controller-manager-859d7b78bd-6strx\" (UID: \"4bdb3c6e-7e04-45a1-9273-04618f0a39cb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:42.958340 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:42.958311 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqww7\" (UniqueName: \"kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7\") pod \"limitador-operator-controller-manager-859d7b78bd-6strx\" (UID: \"4bdb3c6e-7e04-45a1-9273-04618f0a39cb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:43.130649 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.130562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:43.287374 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.287260 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx"] Jul 24 22:42:43.291128 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.291102 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" event={"ID":"d0edc10f-ba97-4554-ba7a-87dd77e36654","Type":"ContainerStarted","Data":"32501a7309f6482ab25c1e368f30a6382fcfbaf576779251db7e55413d34f78c"} Jul 24 22:42:43.291270 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.291237 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:43.292998 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.292975 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" event={"ID":"6e02b98e-82b9-4277-82c4-cca7bdbae251","Type":"ContainerStarted","Data":"5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726"} Jul 24 22:42:43.293116 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.293071 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:43.294118 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.294100 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" event={"ID":"cbd2bd20-e9ce-4d7e-897c-f23122eb370a","Type":"ContainerStarted","Data":"1fb0045bd60f74caabed6e17f98026fff8af7563248510b15d9d0d20d6cb8807"} Jul 24 22:42:43.301968 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:43.301947 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdb3c6e_7e04_45a1_9273_04618f0a39cb.slice/crio-6db890bfae17a02d8734beb45e7a075b2558b95d7f0a0c94a13376a0c16b2930 WatchSource:0}: Error finding container 6db890bfae17a02d8734beb45e7a075b2558b95d7f0a0c94a13376a0c16b2930: Status 404 returned error can't find the container with id 6db890bfae17a02d8734beb45e7a075b2558b95d7f0a0c94a13376a0c16b2930 Jul 24 22:42:43.328081 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.328034 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" podStartSLOduration=1.740845239 podStartE2EDuration="5.328020437s" totalImagesPullingTime="3.587175198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:43.326568954 +0000 UTC m=+426.533807764" watchObservedRunningTime="2026-07-24 22:42:43.328020437 +0000 UTC m=+426.535259245" Jul 24 22:42:43.357641 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:43.357595 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" podStartSLOduration=2.226315129 podStartE2EDuration="8.357578665s" totalImagesPullingTime="6.131263536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:43.356803572 +0000 UTC m=+426.564042392" watchObservedRunningTime="2026-07-24 22:42:43.357578665 +0000 UTC m=+426.564817474" Jul 24 22:42:44.301816 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:44.301776 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" event={"ID":"4bdb3c6e-7e04-45a1-9273-04618f0a39cb","Type":"ContainerStarted","Data":"6db890bfae17a02d8734beb45e7a075b2558b95d7f0a0c94a13376a0c16b2930"} Jul 24 22:42:45.307284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.307174 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" event={"ID":"4bdb3c6e-7e04-45a1-9273-04618f0a39cb","Type":"ContainerStarted","Data":"c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a"} Jul 24 22:42:45.307284 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.307273 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:45.308859 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.308836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" event={"ID":"cbd2bd20-e9ce-4d7e-897c-f23122eb370a","Type":"ContainerStarted","Data":"b415e18c659fb8942337b6d27d58ff69760c097ddb7d755bbf25b7f7610a432f"} Jul 24 22:42:45.308983 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.308951 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:45.330490 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.330441 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" podStartSLOduration=1.632881483 podStartE2EDuration="3.330424816s" totalImagesPullingTime="1.697543333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:45.32784911 +0000 UTC m=+428.535087930" watchObservedRunningTime="2026-07-24 22:42:45.330424816 +0000 UTC m=+428.537663627" Jul 24 22:42:45.360597 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.360539 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" podStartSLOduration=2.777983132 podStartE2EDuration="5.360517849s" totalImagesPullingTime="2.582534717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:42:45.360252619 +0000 UTC m=+428.567491428" watchObservedRunningTime="2026-07-24 22:42:45.360517849 +0000 UTC m=+428.567756657" Jul 24 22:42:45.420446 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.420413 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p"] Jul 24 22:42:45.423897 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.423866 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.428594 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.428563 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tlvrr\"" Jul 24 22:42:45.428594 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.428576 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 22:42:45.428812 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.428583 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 22:42:45.436496 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.436475 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p"] Jul 24 22:42:45.471399 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.471371 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-kube-api-access-gttqw\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.471518 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.471499 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.471571 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.471556 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.572150 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.572067 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.572150 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.572121 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-kube-api-access-gttqw\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.572377 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.572209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.572852 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.572827 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.574681 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.574659 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.588118 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.588088 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gttqw\" (UniqueName: \"kubernetes.io/projected/8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07-kube-api-access-gttqw\") pod \"kuadrant-console-plugin-66b5774985-9tl2p\" (UID: \"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.736331 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.736303 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" Jul 24 22:42:45.872125 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:45.872103 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p"] Jul 24 22:42:45.873911 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:42:45.873879 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ecae87b_9caa_4ec6_9b10_f2d74ffe4e07.slice/crio-968c4ebb2c3c185790f6887a386117d8c980496c22da09a696451f65172f32de WatchSource:0}: Error finding container 968c4ebb2c3c185790f6887a386117d8c980496c22da09a696451f65172f32de: Status 404 returned error can't find the container with id 968c4ebb2c3c185790f6887a386117d8c980496c22da09a696451f65172f32de Jul 24 22:42:46.314944 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:46.314854 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" event={"ID":"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07","Type":"ContainerStarted","Data":"968c4ebb2c3c185790f6887a386117d8c980496c22da09a696451f65172f32de"} Jul 24 22:42:54.304714 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:54.304679 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:54.305154 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:54.304774 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-srzdk" Jul 24 22:42:56.128520 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.128491 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4"] Jul 24 22:42:56.128967 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.128702 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" containerName="manager" containerID="cri-o://5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726" gracePeriod=2 Jul 24 22:42:56.139113 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.139088 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4"] Jul 24 22:42:56.149105 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.149065 2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6e02b98e-82b9-4277-82c4-cca7bdbae251\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\": pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:56.157059 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.157025 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:42:56.157751 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.157731 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" containerName="manager" Jul 24 22:42:56.157859 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.157761 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" containerName="manager" Jul 24 22:42:56.157911 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.157867 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" containerName="manager" Jul 24 22:42:56.179458 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.179420 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx"] Jul 24 22:42:56.179591 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.179472 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx"] Jul 24 22:42:56.179591 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.179526 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:42:56.179726 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.179610 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.179779 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.179728 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" containerName="manager" containerID="cri-o://c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a" gracePeriod=2 Jul 24 22:42:56.182406 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.182251 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:56.184499 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.184467 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6strx\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:56.186617 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.186576 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk"] Jul 24 22:42:56.187157 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.187127 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" containerName="manager" Jul 24 22:42:56.187157 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.187149 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" containerName="manager" Jul 24 22:42:56.187327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.187304 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" containerName="manager" Jul 24 22:42:56.195259 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.194477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:42:56.212456 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.212421 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk"] Jul 24 22:42:56.212577 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.212425 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:56.217695 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.216784 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6strx\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:56.218960 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.218933 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:56.221306 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.221280 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6strx\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:56.270562 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.270536 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.270706 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.270582 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c4l8\" (UniqueName: \"kubernetes.io/projected/27910791-96a1-44b8-a523-21c10f56c6fa-kube-api-access-5c4l8\") pod \"limitador-operator-controller-manager-859d7b78bd-pnhtk\" (UID: \"27910791-96a1-44b8-a523-21c10f56c6fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:42:56.270706 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.270609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc2b6\" (UniqueName: \"kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.317470 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.317448 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-84c6b" Jul 24 22:42:56.320240 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.320203 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:56.322453 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:56.322406 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6strx\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:42:56.371567 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.371532 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.371724 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.371610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c4l8\" (UniqueName: \"kubernetes.io/projected/27910791-96a1-44b8-a523-21c10f56c6fa-kube-api-access-5c4l8\") pod \"limitador-operator-controller-manager-859d7b78bd-pnhtk\" (UID: \"27910791-96a1-44b8-a523-21c10f56c6fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:42:56.371724 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.371639 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc2b6\" (UniqueName: \"kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.373215 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.371928 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.395070 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.395003 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c4l8\" (UniqueName: \"kubernetes.io/projected/27910791-96a1-44b8-a523-21c10f56c6fa-kube-api-access-5c4l8\") pod \"limitador-operator-controller-manager-859d7b78bd-pnhtk\" (UID: \"27910791-96a1-44b8-a523-21c10f56c6fa\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:42:56.395545 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.395523 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc2b6\" (UniqueName: \"kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wlnvz\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.604601 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.604566 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:42:56.610737 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:42:56.610711 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:42:57.212980 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:57.212935 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kzvr4\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:42:57.215164 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:42:57.215128 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6strx\" is forbidden: User \"system:node:ip-10-0-130-123.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-123.ec2.internal' and this object" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:43:07.020473 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.020329 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:43:07.026352 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.026328 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:43:07.048137 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.048110 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:43:07.052642 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:43:07.052613 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c45951b_c240_49be_9b64_9e5cd4eacf5f.slice/crio-86314b80f435f1b0527c8d2d201bba321a3676b4b55cffc20ca3452cb966fba7 WatchSource:0}: Error finding container 86314b80f435f1b0527c8d2d201bba321a3676b4b55cffc20ca3452cb966fba7: Status 404 returned error can't find the container with id 86314b80f435f1b0527c8d2d201bba321a3676b4b55cffc20ca3452cb966fba7 Jul 24 22:43:07.072363 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.072339 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk"] Jul 24 22:43:07.170126 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.170103 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume\") pod \"6e02b98e-82b9-4277-82c4-cca7bdbae251\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " Jul 24 22:43:07.170267 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.170184 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqww7\" (UniqueName: \"kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7\") pod \"4bdb3c6e-7e04-45a1-9273-04618f0a39cb\" (UID: \"4bdb3c6e-7e04-45a1-9273-04618f0a39cb\") " Jul 24 22:43:07.170267 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.170204 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdmq5\" (UniqueName: \"kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5\") pod \"6e02b98e-82b9-4277-82c4-cca7bdbae251\" (UID: \"6e02b98e-82b9-4277-82c4-cca7bdbae251\") " Jul 24 22:43:07.170607 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.170578 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6e02b98e-82b9-4277-82c4-cca7bdbae251" (UID: "6e02b98e-82b9-4277-82c4-cca7bdbae251"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:43:07.172363 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.172342 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5" (OuterVolumeSpecName: "kube-api-access-jdmq5") pod "6e02b98e-82b9-4277-82c4-cca7bdbae251" (UID: "6e02b98e-82b9-4277-82c4-cca7bdbae251"). InnerVolumeSpecName "kube-api-access-jdmq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:07.172363 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.172357 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7" (OuterVolumeSpecName: "kube-api-access-nqww7") pod "4bdb3c6e-7e04-45a1-9273-04618f0a39cb" (UID: "4bdb3c6e-7e04-45a1-9273-04618f0a39cb"). InnerVolumeSpecName "kube-api-access-nqww7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:07.213931 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.213904 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" path="/var/lib/kubelet/pods/4bdb3c6e-7e04-45a1-9273-04618f0a39cb/volumes" Jul 24 22:43:07.214268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.214254 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e02b98e-82b9-4277-82c4-cca7bdbae251" path="/var/lib/kubelet/pods/6e02b98e-82b9-4277-82c4-cca7bdbae251/volumes" Jul 24 22:43:07.271395 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.271368 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqww7\" (UniqueName: \"kubernetes.io/projected/4bdb3c6e-7e04-45a1-9273-04618f0a39cb-kube-api-access-nqww7\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:07.271395 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.271391 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdmq5\" (UniqueName: \"kubernetes.io/projected/6e02b98e-82b9-4277-82c4-cca7bdbae251-kube-api-access-jdmq5\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:07.271395 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.271402 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6e02b98e-82b9-4277-82c4-cca7bdbae251-extensions-socket-volume\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:07.434676 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.434581 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" event={"ID":"27910791-96a1-44b8-a523-21c10f56c6fa","Type":"ContainerStarted","Data":"448e7ba093f4a0184ccfe08bb7933dd3956c080f737afee7783c240770829710"} Jul 24 22:43:07.434676 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.434645 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" event={"ID":"27910791-96a1-44b8-a523-21c10f56c6fa","Type":"ContainerStarted","Data":"a0e77ec1bf2ce5d3abb9c1e84171896290be6c3573848d5c26eb9e49927e5d5b"} Jul 24 22:43:07.434866 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.434677 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:43:07.435902 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.435863 2559 generic.go:356] "Generic (PLEG): container finished" podID="4bdb3c6e-7e04-45a1-9273-04618f0a39cb" containerID="c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a" exitCode=0 Jul 24 22:43:07.436030 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.435908 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6strx" Jul 24 22:43:07.436030 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.435949 2559 scope.go:117] "RemoveContainer" containerID="c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a" Jul 24 22:43:07.437847 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.437819 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" event={"ID":"8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07","Type":"ContainerStarted","Data":"a3ad387acf67e095f66b72d49cc3e7aa1225f069675ac4b9abe87e293dd7d687"} Jul 24 22:43:07.440067 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.440044 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" event={"ID":"0c45951b-c240-49be-9b64-9e5cd4eacf5f","Type":"ContainerStarted","Data":"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed"} Jul 24 22:43:07.440178 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.440091 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" event={"ID":"0c45951b-c240-49be-9b64-9e5cd4eacf5f","Type":"ContainerStarted","Data":"86314b80f435f1b0527c8d2d201bba321a3676b4b55cffc20ca3452cb966fba7"} Jul 24 22:43:07.440178 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.440144 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:43:07.441448 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.441429 2559 generic.go:356] "Generic (PLEG): container finished" podID="6e02b98e-82b9-4277-82c4-cca7bdbae251" containerID="5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726" exitCode=0 Jul 24 22:43:07.441531 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.441484 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kzvr4" Jul 24 22:43:07.445508 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.445491 2559 scope.go:117] "RemoveContainer" containerID="c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a" Jul 24 22:43:07.445768 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:43:07.445749 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a\": container with ID starting with c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a not found: ID does not exist" containerID="c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a" Jul 24 22:43:07.445814 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.445779 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a"} err="failed to get container status \"c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a\": rpc error: code = NotFound desc = could not find container \"c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a\": container with ID starting with c02f48afddeebcb1fba7cf0b4a2d50b706ba218c6ad29e5d072eff71248fc99a not found: ID does not exist" Jul 24 22:43:07.445814 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.445795 2559 scope.go:117] "RemoveContainer" containerID="5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726" Jul 24 22:43:07.454135 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.454118 2559 scope.go:117] "RemoveContainer" containerID="5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726" Jul 24 22:43:07.454518 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:43:07.454495 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726\": container with ID starting with 5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726 not found: ID does not exist" containerID="5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726" Jul 24 22:43:07.454655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.454625 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726"} err="failed to get container status \"5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726\": rpc error: code = NotFound desc = could not find container \"5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726\": container with ID starting with 5dd1d4a121e640dc017b865447755a25a44cf47747094927bd38274c9364c726 not found: ID does not exist" Jul 24 22:43:07.455673 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.455636 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" podStartSLOduration=11.455624745 podStartE2EDuration="11.455624745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:07.454143711 +0000 UTC m=+450.661382521" watchObservedRunningTime="2026-07-24 22:43:07.455624745 +0000 UTC m=+450.662863552" Jul 24 22:43:07.472846 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.472807 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9tl2p" podStartSLOduration=1.375076418 podStartE2EDuration="22.472796522s" totalImagesPullingTime="21.097720104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:07.471431359 +0000 UTC m=+450.678670166" watchObservedRunningTime="2026-07-24 22:43:07.472796522 +0000 UTC m=+450.680035329" Jul 24 22:43:07.489529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:07.489484 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" podStartSLOduration=11.489474123 podStartE2EDuration="11.489474123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:07.488819371 +0000 UTC m=+450.696058180" watchObservedRunningTime="2026-07-24 22:43:07.489474123 +0000 UTC m=+450.696712933" Jul 24 22:43:12.539461 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:12.539425 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:43:12.540025 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:12.539620 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" podUID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" containerName="manager" containerID="cri-o://786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed" gracePeriod=10 Jul 24 22:43:12.541705 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:12.541685 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:43:13.361342 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.361315 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:43:13.474583 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.474505 2559 generic.go:356] "Generic (PLEG): container finished" podID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" containerID="786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed" exitCode=0 Jul 24 22:43:13.474583 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.474556 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" Jul 24 22:43:13.474791 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.474587 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" event={"ID":"0c45951b-c240-49be-9b64-9e5cd4eacf5f","Type":"ContainerDied","Data":"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed"} Jul 24 22:43:13.474791 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.474639 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz" event={"ID":"0c45951b-c240-49be-9b64-9e5cd4eacf5f","Type":"ContainerDied","Data":"86314b80f435f1b0527c8d2d201bba321a3676b4b55cffc20ca3452cb966fba7"} Jul 24 22:43:13.474791 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.474662 2559 scope.go:117] "RemoveContainer" containerID="786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed" Jul 24 22:43:13.483317 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.483300 2559 scope.go:117] "RemoveContainer" containerID="786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed" Jul 24 22:43:13.483568 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:43:13.483550 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed\": container with ID starting with 786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed not found: ID does not exist" containerID="786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed" Jul 24 22:43:13.483632 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.483577 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed"} err="failed to get container status \"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed\": rpc error: code = NotFound desc = could not find container \"786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed\": container with ID starting with 786b7dcdd1336a9edcff264dcc46497e06faadfeab3067fb38c97b4436d0f0ed not found: ID does not exist" Jul 24 22:43:13.522477 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.522452 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume\") pod \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " Jul 24 22:43:13.522578 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.522527 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc2b6\" (UniqueName: \"kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6\") pod \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\" (UID: \"0c45951b-c240-49be-9b64-9e5cd4eacf5f\") " Jul 24 22:43:13.522788 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.522768 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0c45951b-c240-49be-9b64-9e5cd4eacf5f" (UID: "0c45951b-c240-49be-9b64-9e5cd4eacf5f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:43:13.524720 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.524695 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6" (OuterVolumeSpecName: "kube-api-access-tc2b6") pod "0c45951b-c240-49be-9b64-9e5cd4eacf5f" (UID: "0c45951b-c240-49be-9b64-9e5cd4eacf5f"). InnerVolumeSpecName "kube-api-access-tc2b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:13.623823 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.623796 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0c45951b-c240-49be-9b64-9e5cd4eacf5f-extensions-socket-volume\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:13.623823 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.623820 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc2b6\" (UniqueName: \"kubernetes.io/projected/0c45951b-c240-49be-9b64-9e5cd4eacf5f-kube-api-access-tc2b6\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:13.804889 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.804854 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:43:13.810655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:13.810622 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wlnvz"] Jul 24 22:43:15.213837 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:15.213805 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" path="/var/lib/kubelet/pods/0c45951b-c240-49be-9b64-9e5cd4eacf5f/volumes" Jul 24 22:43:18.451126 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:18.451092 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pnhtk" Jul 24 22:43:30.662831 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.662803 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:30.663182 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.663148 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" containerName="manager" Jul 24 22:43:30.663182 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.663158 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" containerName="manager" Jul 24 22:43:30.663267 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.663239 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c45951b-c240-49be-9b64-9e5cd4eacf5f" containerName="manager" Jul 24 22:43:30.666500 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.666477 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.669196 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.669170 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 22:43:30.674624 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.674601 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:30.759455 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.759419 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:30.766789 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.766763 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.766958 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.766932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrqm9\" (UniqueName: \"kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.868111 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.868071 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrqm9\" (UniqueName: \"kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.868306 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.868145 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.868804 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.868780 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.878870 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.878837 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrqm9\" (UniqueName: \"kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9\") pod \"limitador-limitador-64c94fd64f-st6tg\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:30.977018 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:30.976932 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:31.113456 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.113432 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:31.115511 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:43:31.115474 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3054c802_c28d_4fe3_98cf_c6797ee2f9c4.slice/crio-a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1 WatchSource:0}: Error finding container a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1: Status 404 returned error can't find the container with id a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1 Jul 24 22:43:31.559484 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.559446 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" event={"ID":"3054c802-c28d-4fe3-98cf-c6797ee2f9c4","Type":"ContainerStarted","Data":"a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1"} Jul 24 22:43:31.702801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.702767 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:43:31.707405 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.706621 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:43:31.709674 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.709648 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4ktgx\"" Jul 24 22:43:31.714972 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.714944 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:43:31.777472 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.777167 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwxg\" (UniqueName: \"kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg\") pod \"authorino-75d556bd77-598l5\" (UID: \"8b604953-d455-4bca-befe-afda2a65ab4c\") " pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:43:31.879089 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.878872 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwxg\" (UniqueName: \"kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg\") pod \"authorino-75d556bd77-598l5\" (UID: \"8b604953-d455-4bca-befe-afda2a65ab4c\") " pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:43:31.896358 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:31.896321 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwxg\" (UniqueName: \"kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg\") pod \"authorino-75d556bd77-598l5\" (UID: \"8b604953-d455-4bca-befe-afda2a65ab4c\") " pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:43:32.021297 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:32.021266 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:43:32.175597 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:32.175562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:43:32.177562 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:43:32.177534 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b604953_d455_4bca_befe_afda2a65ab4c.slice/crio-7e76f3144e20bd6cdd75d7073cc03c131e51cf338dc1b87749fe3d121e08a6d6 WatchSource:0}: Error finding container 7e76f3144e20bd6cdd75d7073cc03c131e51cf338dc1b87749fe3d121e08a6d6: Status 404 returned error can't find the container with id 7e76f3144e20bd6cdd75d7073cc03c131e51cf338dc1b87749fe3d121e08a6d6 Jul 24 22:43:32.566365 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:32.566286 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-598l5" event={"ID":"8b604953-d455-4bca-befe-afda2a65ab4c","Type":"ContainerStarted","Data":"7e76f3144e20bd6cdd75d7073cc03c131e51cf338dc1b87749fe3d121e08a6d6"} Jul 24 22:43:34.580762 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:34.580723 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" event={"ID":"3054c802-c28d-4fe3-98cf-c6797ee2f9c4","Type":"ContainerStarted","Data":"643737ee72e13e08bf288840c873e75e6b11beb3928917e137b7b8623c86e51e"} Jul 24 22:43:34.581214 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:34.580788 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:36.593479 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:36.593443 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-598l5" event={"ID":"8b604953-d455-4bca-befe-afda2a65ab4c","Type":"ContainerStarted","Data":"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3"} Jul 24 22:43:36.610281 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:36.610202 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" podStartSLOduration=3.921462255 podStartE2EDuration="6.610187618s" totalImagesPullingTime="2.688725363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:34.609757723 +0000 UTC m=+477.816996548" watchObservedRunningTime="2026-07-24 22:43:36.610187618 +0000 UTC m=+479.817426424" Jul 24 22:43:36.611060 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:36.611031 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-598l5" podStartSLOduration=1.881654346 podStartE2EDuration="5.611021055s" totalImagesPullingTime="3.729366709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:36.609802364 +0000 UTC m=+479.817041170" watchObservedRunningTime="2026-07-24 22:43:36.611021055 +0000 UTC m=+479.818259861" Jul 24 22:43:45.587913 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:45.587883 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:45.694338 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:45.694307 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:45.694511 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:45.694458 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" podUID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" containerName="limitador" containerID="cri-o://643737ee72e13e08bf288840c873e75e6b11beb3928917e137b7b8623c86e51e" gracePeriod=30 Jul 24 22:43:46.640277 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.640218 2559 generic.go:356] "Generic (PLEG): container finished" podID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" containerID="643737ee72e13e08bf288840c873e75e6b11beb3928917e137b7b8623c86e51e" exitCode=0 Jul 24 22:43:46.640277 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.640257 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" event={"ID":"3054c802-c28d-4fe3-98cf-c6797ee2f9c4","Type":"ContainerDied","Data":"643737ee72e13e08bf288840c873e75e6b11beb3928917e137b7b8623c86e51e"} Jul 24 22:43:46.640607 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.640312 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" event={"ID":"3054c802-c28d-4fe3-98cf-c6797ee2f9c4","Type":"ContainerDied","Data":"a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1"} Jul 24 22:43:46.640607 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.640332 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5889946e92b79bb6ca58f3daf01d25b166492fcfa2d9a99577f6b5cc19916f1" Jul 24 22:43:46.648693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.648676 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:46.705766 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.705743 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrqm9\" (UniqueName: \"kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9\") pod \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " Jul 24 22:43:46.705918 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.705803 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file\") pod \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\" (UID: \"3054c802-c28d-4fe3-98cf-c6797ee2f9c4\") " Jul 24 22:43:46.706138 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.706117 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file" (OuterVolumeSpecName: "config-file") pod "3054c802-c28d-4fe3-98cf-c6797ee2f9c4" (UID: "3054c802-c28d-4fe3-98cf-c6797ee2f9c4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:43:46.707838 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.707821 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9" (OuterVolumeSpecName: "kube-api-access-zrqm9") pod "3054c802-c28d-4fe3-98cf-c6797ee2f9c4" (UID: "3054c802-c28d-4fe3-98cf-c6797ee2f9c4"). InnerVolumeSpecName "kube-api-access-zrqm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:46.807022 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.806966 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrqm9\" (UniqueName: \"kubernetes.io/projected/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-kube-api-access-zrqm9\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:46.807022 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:46.806990 2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3054c802-c28d-4fe3-98cf-c6797ee2f9c4-config-file\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:43:47.345269 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.345239 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m9v72"] Jul 24 22:43:47.345613 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.345601 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" containerName="limitador" Jul 24 22:43:47.345655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.345618 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" containerName="limitador" Jul 24 22:43:47.345693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.345684 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" containerName="limitador" Jul 24 22:43:47.349055 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.349040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.351999 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.351974 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 24 22:43:47.352133 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.352058 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 24 22:43:47.352280 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.352193 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xlvg5\"" Jul 24 22:43:47.352280 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.352197 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 22:43:47.359963 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.359944 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m9v72"] Jul 24 22:43:47.411235 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.411202 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n7zc\" (UniqueName: \"kubernetes.io/projected/28bdfb18-1435-4308-889e-1f60688bdd3d-kube-api-access-4n7zc\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.411335 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.411249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28bdfb18-1435-4308-889e-1f60688bdd3d-data\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.512036 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.511971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n7zc\" (UniqueName: \"kubernetes.io/projected/28bdfb18-1435-4308-889e-1f60688bdd3d-kube-api-access-4n7zc\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.512036 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.512001 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28bdfb18-1435-4308-889e-1f60688bdd3d-data\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.512358 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.512343 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28bdfb18-1435-4308-889e-1f60688bdd3d-data\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.521736 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.521714 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n7zc\" (UniqueName: \"kubernetes.io/projected/28bdfb18-1435-4308-889e-1f60688bdd3d-kube-api-access-4n7zc\") pod \"postgres-69fb4594cf-m9v72\" (UID: \"28bdfb18-1435-4308-889e-1f60688bdd3d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.644984 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.644959 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-st6tg" Jul 24 22:43:47.659660 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.659637 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:47.666563 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.666543 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:47.671607 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.671589 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-st6tg"] Jul 24 22:43:47.796829 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:47.796804 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-m9v72"] Jul 24 22:43:47.798934 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:43:47.798907 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bdfb18_1435_4308_889e_1f60688bdd3d.slice/crio-a1584baadbcecd10cb742791885b539bc35abb057385b6f7751759072a339556 WatchSource:0}: Error finding container a1584baadbcecd10cb742791885b539bc35abb057385b6f7751759072a339556: Status 404 returned error can't find the container with id a1584baadbcecd10cb742791885b539bc35abb057385b6f7751759072a339556 Jul 24 22:43:48.654097 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:48.654026 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" event={"ID":"28bdfb18-1435-4308-889e-1f60688bdd3d","Type":"ContainerStarted","Data":"a1584baadbcecd10cb742791885b539bc35abb057385b6f7751759072a339556"} Jul 24 22:43:49.216641 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:49.216606 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3054c802-c28d-4fe3-98cf-c6797ee2f9c4" path="/var/lib/kubelet/pods/3054c802-c28d-4fe3-98cf-c6797ee2f9c4/volumes" Jul 24 22:43:53.683809 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:53.683774 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" event={"ID":"28bdfb18-1435-4308-889e-1f60688bdd3d","Type":"ContainerStarted","Data":"c5f775547a2c0bb47d1f927ce2e0dc90dc66c8a78cce18c473701666c3e97a0e"} Jul 24 22:43:53.684179 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:53.683837 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:43:53.701458 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:53.701412 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" podStartSLOduration=1.814417959 podStartE2EDuration="6.701395819s" totalImagesPullingTime="4.88697786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:53.700395209 +0000 UTC m=+496.907634019" watchObservedRunningTime="2026-07-24 22:43:53.701395819 +0000 UTC m=+496.908634627" Jul 24 22:43:59.720094 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:43:59.720061 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-m9v72" Jul 24 22:44:00.286305 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.286275 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:00.290008 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.289988 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:00.297326 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.297304 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:00.418302 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.418270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5hvk\" (UniqueName: \"kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk\") pod \"authorino-5d4b67c446-mtgn6\" (UID: \"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6\") " pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:00.519319 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.519294 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hvk\" (UniqueName: \"kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk\") pod \"authorino-5d4b67c446-mtgn6\" (UID: \"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6\") " pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:00.528119 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.528096 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hvk\" (UniqueName: \"kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk\") pod \"authorino-5d4b67c446-mtgn6\" (UID: \"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6\") " pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:00.538017 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.537965 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:00.538148 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.538137 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:00.566464 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.566441 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:00.570185 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.570167 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:00.577528 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.577499 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:00.620609 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.620583 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hns58\" (UniqueName: \"kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58\") pod \"authorino-85d58f796b-xkj29\" (UID: \"5e9ee997-3ede-4af7-8ef1-303f6384d573\") " pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:00.677022 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.676969 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:00.679400 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:00.679376 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e4f0104_a0b5_48f5_90bd_2df35c6bc1a6.slice/crio-b9f1c8d26c861fc680bdbca542e96121618caf5e1ab11eb1cbd0b7bafc055582 WatchSource:0}: Error finding container b9f1c8d26c861fc680bdbca542e96121618caf5e1ab11eb1cbd0b7bafc055582: Status 404 returned error can't find the container with id b9f1c8d26c861fc680bdbca542e96121618caf5e1ab11eb1cbd0b7bafc055582 Jul 24 22:44:00.721078 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.721047 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" event={"ID":"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6","Type":"ContainerStarted","Data":"b9f1c8d26c861fc680bdbca542e96121618caf5e1ab11eb1cbd0b7bafc055582"} Jul 24 22:44:00.721401 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.721059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hns58\" (UniqueName: \"kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58\") pod \"authorino-85d58f796b-xkj29\" (UID: \"5e9ee997-3ede-4af7-8ef1-303f6384d573\") " pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:00.732444 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.732422 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hns58\" (UniqueName: \"kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58\") pod \"authorino-85d58f796b-xkj29\" (UID: \"5e9ee997-3ede-4af7-8ef1-303f6384d573\") " pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:00.881626 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:00.881606 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:01.009381 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.009361 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:01.011462 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:01.011431 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e9ee997_3ede_4af7_8ef1_303f6384d573.slice/crio-c3465c849b0cde8f07562f8bf228719c2de6df5292bfcec38924a8a7503d1b68 WatchSource:0}: Error finding container c3465c849b0cde8f07562f8bf228719c2de6df5292bfcec38924a8a7503d1b68: Status 404 returned error can't find the container with id c3465c849b0cde8f07562f8bf228719c2de6df5292bfcec38924a8a7503d1b68 Jul 24 22:44:01.727258 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.727149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" event={"ID":"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6","Type":"ContainerStarted","Data":"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7"} Jul 24 22:44:01.727258 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.727196 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" podUID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" containerName="authorino" containerID="cri-o://6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7" gracePeriod=30 Jul 24 22:44:01.728778 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.728753 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d58f796b-xkj29" event={"ID":"5e9ee997-3ede-4af7-8ef1-303f6384d573","Type":"ContainerStarted","Data":"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552"} Jul 24 22:44:01.728896 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.728791 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d58f796b-xkj29" event={"ID":"5e9ee997-3ede-4af7-8ef1-303f6384d573","Type":"ContainerStarted","Data":"c3465c849b0cde8f07562f8bf228719c2de6df5292bfcec38924a8a7503d1b68"} Jul 24 22:44:01.745859 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.745819 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" podStartSLOduration=1.322257606 podStartE2EDuration="1.745807208s" totalImagesPullingTime="423.549602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:01.744294015 +0000 UTC m=+504.951532823" watchObservedRunningTime="2026-07-24 22:44:01.745807208 +0000 UTC m=+504.953046015" Jul 24 22:44:01.764150 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.764109 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85d58f796b-xkj29" podStartSLOduration=1.391731914 podStartE2EDuration="1.764097053s" totalImagesPullingTime="372.365139ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:01.763550222 +0000 UTC m=+504.970789041" watchObservedRunningTime="2026-07-24 22:44:01.764097053 +0000 UTC m=+504.971335910" Jul 24 22:44:01.806727 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.806620 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:44:01.806987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:01.806927 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-598l5" podUID="8b604953-d455-4bca-befe-afda2a65ab4c" containerName="authorino" containerID="cri-o://5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3" gracePeriod=30 Jul 24 22:44:02.040896 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.040866 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-69bc55c65d-zs5kt"] Jul 24 22:44:02.046077 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.046057 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.048897 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.048878 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 22:44:02.052895 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.052874 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69bc55c65d-zs5kt"] Jul 24 22:44:02.094424 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.094404 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:02.097053 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.097027 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:44:02.133951 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.133927 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpxq6\" (UniqueName: \"kubernetes.io/projected/7a688ba1-11bb-48aa-abda-7f8101dee067-kube-api-access-kpxq6\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.134077 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.134038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a688ba1-11bb-48aa-abda-7f8101dee067-tls-cert\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.234923 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.234896 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5hvk\" (UniqueName: \"kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk\") pod \"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6\" (UID: \"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6\") " Jul 24 22:44:02.235041 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.234940 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdwxg\" (UniqueName: \"kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg\") pod \"8b604953-d455-4bca-befe-afda2a65ab4c\" (UID: \"8b604953-d455-4bca-befe-afda2a65ab4c\") " Jul 24 22:44:02.235083 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.235066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a688ba1-11bb-48aa-abda-7f8101dee067-tls-cert\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.235161 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.235144 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpxq6\" (UniqueName: \"kubernetes.io/projected/7a688ba1-11bb-48aa-abda-7f8101dee067-kube-api-access-kpxq6\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.237183 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.237153 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg" (OuterVolumeSpecName: "kube-api-access-jdwxg") pod "8b604953-d455-4bca-befe-afda2a65ab4c" (UID: "8b604953-d455-4bca-befe-afda2a65ab4c"). InnerVolumeSpecName "kube-api-access-jdwxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:02.237307 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.237189 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk" (OuterVolumeSpecName: "kube-api-access-q5hvk") pod "1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" (UID: "1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6"). InnerVolumeSpecName "kube-api-access-q5hvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:02.237511 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.237494 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a688ba1-11bb-48aa-abda-7f8101dee067-tls-cert\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.243764 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.243746 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpxq6\" (UniqueName: \"kubernetes.io/projected/7a688ba1-11bb-48aa-abda-7f8101dee067-kube-api-access-kpxq6\") pod \"authorino-69bc55c65d-zs5kt\" (UID: \"7a688ba1-11bb-48aa-abda-7f8101dee067\") " pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.335756 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.335705 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5hvk\" (UniqueName: \"kubernetes.io/projected/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6-kube-api-access-q5hvk\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:44:02.335756 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.335725 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdwxg\" (UniqueName: \"kubernetes.io/projected/8b604953-d455-4bca-befe-afda2a65ab4c-kube-api-access-jdwxg\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:44:02.357637 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.357618 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-69bc55c65d-zs5kt" Jul 24 22:44:02.532877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.532856 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-69bc55c65d-zs5kt"] Jul 24 22:44:02.534694 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:02.534660 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a688ba1_11bb_48aa_abda_7f8101dee067.slice/crio-f509ddad8857716b7192c937941ef7f9d25a65278a3338f1f189df6ee6339dc3 WatchSource:0}: Error finding container f509ddad8857716b7192c937941ef7f9d25a65278a3338f1f189df6ee6339dc3: Status 404 returned error can't find the container with id f509ddad8857716b7192c937941ef7f9d25a65278a3338f1f189df6ee6339dc3 Jul 24 22:44:02.735733 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.735697 2559 generic.go:356] "Generic (PLEG): container finished" podID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" containerID="6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7" exitCode=0 Jul 24 22:44:02.736159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.735740 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" Jul 24 22:44:02.736159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.735779 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" event={"ID":"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6","Type":"ContainerDied","Data":"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7"} Jul 24 22:44:02.736159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.735817 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mtgn6" event={"ID":"1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6","Type":"ContainerDied","Data":"b9f1c8d26c861fc680bdbca542e96121618caf5e1ab11eb1cbd0b7bafc055582"} Jul 24 22:44:02.736159 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.735832 2559 scope.go:117] "RemoveContainer" containerID="6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7" Jul 24 22:44:02.737135 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.737087 2559 generic.go:356] "Generic (PLEG): container finished" podID="8b604953-d455-4bca-befe-afda2a65ab4c" containerID="5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3" exitCode=0 Jul 24 22:44:02.737282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.737134 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-598l5" Jul 24 22:44:02.737282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.737176 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-598l5" event={"ID":"8b604953-d455-4bca-befe-afda2a65ab4c","Type":"ContainerDied","Data":"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3"} Jul 24 22:44:02.737282 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.737202 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-598l5" event={"ID":"8b604953-d455-4bca-befe-afda2a65ab4c","Type":"ContainerDied","Data":"7e76f3144e20bd6cdd75d7073cc03c131e51cf338dc1b87749fe3d121e08a6d6"} Jul 24 22:44:02.738663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.738634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bc55c65d-zs5kt" event={"ID":"7a688ba1-11bb-48aa-abda-7f8101dee067","Type":"ContainerStarted","Data":"f509ddad8857716b7192c937941ef7f9d25a65278a3338f1f189df6ee6339dc3"} Jul 24 22:44:02.746196 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.746183 2559 scope.go:117] "RemoveContainer" containerID="6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7" Jul 24 22:44:02.746504 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:02.746485 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7\": container with ID starting with 6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7 not found: ID does not exist" containerID="6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7" Jul 24 22:44:02.746579 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.746515 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7"} err="failed to get container status \"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7\": rpc error: code = NotFound desc = could not find container \"6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7\": container with ID starting with 6aad7b9bfb6ba4419d1c13dfd60dc86c94cb97dc0d5c3d82e980c61cfab4d4b7 not found: ID does not exist" Jul 24 22:44:02.746579 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.746537 2559 scope.go:117] "RemoveContainer" containerID="5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3" Jul 24 22:44:02.754412 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.754395 2559 scope.go:117] "RemoveContainer" containerID="5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3" Jul 24 22:44:02.754634 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:02.754616 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3\": container with ID starting with 5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3 not found: ID does not exist" containerID="5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3" Jul 24 22:44:02.754682 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.754639 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3"} err="failed to get container status \"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3\": rpc error: code = NotFound desc = could not find container \"5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3\": container with ID starting with 5161e57df0e30710f72233d40f81cf082ed125e7269857553d685b59dd92b4a3 not found: ID does not exist" Jul 24 22:44:02.761313 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.761296 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:02.763676 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.763660 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mtgn6"] Jul 24 22:44:02.777387 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.777356 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:44:02.779316 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:02.779288 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-598l5"] Jul 24 22:44:03.216183 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.216153 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" path="/var/lib/kubelet/pods/1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6/volumes" Jul 24 22:44:03.216554 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.216540 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b604953-d455-4bca-befe-afda2a65ab4c" path="/var/lib/kubelet/pods/8b604953-d455-4bca-befe-afda2a65ab4c/volumes" Jul 24 22:44:03.747074 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.747039 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-69bc55c65d-zs5kt" event={"ID":"7a688ba1-11bb-48aa-abda-7f8101dee067","Type":"ContainerStarted","Data":"d6ee9b967e5a7438f4984469fd48b33a399042ed659eb077d8a5e556738f4047"} Jul 24 22:44:03.766984 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.766941 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-69bc55c65d-zs5kt" podStartSLOduration=1.263368907 podStartE2EDuration="1.766926429s" totalImagesPullingTime="503.557522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:03.764563507 +0000 UTC m=+506.971802306" watchObservedRunningTime="2026-07-24 22:44:03.766926429 +0000 UTC m=+506.974165281" Jul 24 22:44:03.790905 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.790878 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:03.791067 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:03.791044 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-85d58f796b-xkj29" podUID="5e9ee997-3ede-4af7-8ef1-303f6384d573" containerName="authorino" containerID="cri-o://c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552" gracePeriod=30 Jul 24 22:44:04.101143 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.101122 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:04.149954 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.149933 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hns58\" (UniqueName: \"kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58\") pod \"5e9ee997-3ede-4af7-8ef1-303f6384d573\" (UID: \"5e9ee997-3ede-4af7-8ef1-303f6384d573\") " Jul 24 22:44:04.151920 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.151891 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58" (OuterVolumeSpecName: "kube-api-access-hns58") pod "5e9ee997-3ede-4af7-8ef1-303f6384d573" (UID: "5e9ee997-3ede-4af7-8ef1-303f6384d573"). InnerVolumeSpecName "kube-api-access-hns58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:04.251168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.251144 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hns58\" (UniqueName: \"kubernetes.io/projected/5e9ee997-3ede-4af7-8ef1-303f6384d573-kube-api-access-hns58\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:44:04.758204 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.758165 2559 generic.go:356] "Generic (PLEG): container finished" podID="5e9ee997-3ede-4af7-8ef1-303f6384d573" containerID="c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552" exitCode=0 Jul 24 22:44:04.758588 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.758218 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d58f796b-xkj29" Jul 24 22:44:04.758588 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.758254 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d58f796b-xkj29" event={"ID":"5e9ee997-3ede-4af7-8ef1-303f6384d573","Type":"ContainerDied","Data":"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552"} Jul 24 22:44:04.758588 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.758292 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d58f796b-xkj29" event={"ID":"5e9ee997-3ede-4af7-8ef1-303f6384d573","Type":"ContainerDied","Data":"c3465c849b0cde8f07562f8bf228719c2de6df5292bfcec38924a8a7503d1b68"} Jul 24 22:44:04.758588 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.758309 2559 scope.go:117] "RemoveContainer" containerID="c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552" Jul 24 22:44:04.766687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.766673 2559 scope.go:117] "RemoveContainer" containerID="c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552" Jul 24 22:44:04.766924 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:04.766908 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552\": container with ID starting with c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552 not found: ID does not exist" containerID="c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552" Jul 24 22:44:04.766976 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.766930 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552"} err="failed to get container status \"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552\": rpc error: code = NotFound desc = could not find container \"c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552\": container with ID starting with c5e20e9cec5709a0b4d514dc69b00f23e776c604f7e9eaef9ccfb909fd34d552 not found: ID does not exist" Jul 24 22:44:04.778310 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.778286 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:04.780195 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:04.780177 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-85d58f796b-xkj29"] Jul 24 22:44:05.214107 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:05.214080 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e9ee997-3ede-4af7-8ef1-303f6384d573" path="/var/lib/kubelet/pods/5e9ee997-3ede-4af7-8ef1-303f6384d573/volumes" Jul 24 22:44:06.612283 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612253 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:06.612647 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612609 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e9ee997-3ede-4af7-8ef1-303f6384d573" containerName="authorino" Jul 24 22:44:06.612647 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612620 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e9ee997-3ede-4af7-8ef1-303f6384d573" containerName="authorino" Jul 24 22:44:06.612647 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612637 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b604953-d455-4bca-befe-afda2a65ab4c" containerName="authorino" Jul 24 22:44:06.612647 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612642 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b604953-d455-4bca-befe-afda2a65ab4c" containerName="authorino" Jul 24 22:44:06.612774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612656 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" containerName="authorino" Jul 24 22:44:06.612774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612662 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" containerName="authorino" Jul 24 22:44:06.612774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612723 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b604953-d455-4bca-befe-afda2a65ab4c" containerName="authorino" Jul 24 22:44:06.612774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612735 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e9ee997-3ede-4af7-8ef1-303f6384d573" containerName="authorino" Jul 24 22:44:06.612774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.612742 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e4f0104-a0b5-48f5-90bd-2df35c6bc1a6" containerName="authorino" Jul 24 22:44:06.778406 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.778377 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:06.778568 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.778475 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.781646 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.781621 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 22:44:06.781783 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.781671 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tgrvn\"" Jul 24 22:44:06.782920 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.782904 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 22:44:06.792092 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.792073 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79678c6f7-hl89h"] Jul 24 22:44:06.815710 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.815690 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79678c6f7-hl89h"] Jul 24 22:44:06.815806 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.815780 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.869285 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.869197 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.869396 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.869293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcgkw\" (UniqueName: \"kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.869396 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.869358 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4dvj\" (UniqueName: \"kubernetes.io/projected/732e595b-3bfe-414e-9381-83a63f8c4d5c-kube-api-access-f4dvj\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.869483 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.869414 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732e595b-3bfe-414e-9381-83a63f8c4d5c-webhook-cert\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.970553 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.970527 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.970663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.970582 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcgkw\" (UniqueName: \"kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.970663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.970612 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4dvj\" (UniqueName: \"kubernetes.io/projected/732e595b-3bfe-414e-9381-83a63f8c4d5c-kube-api-access-f4dvj\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.970663 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.970643 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732e595b-3bfe-414e-9381-83a63f8c4d5c-webhook-cert\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.973021 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.972990 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/732e595b-3bfe-414e-9381-83a63f8c4d5c-webhook-cert\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:06.973101 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.973068 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.982574 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.982554 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcgkw\" (UniqueName: \"kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw\") pod \"maas-controller-7875b74595-jgltg\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:06.983009 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:06.982993 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4dvj\" (UniqueName: \"kubernetes.io/projected/732e595b-3bfe-414e-9381-83a63f8c4d5c-kube-api-access-f4dvj\") pod \"maas-controller-79678c6f7-hl89h\" (UID: \"732e595b-3bfe-414e-9381-83a63f8c4d5c\") " pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:07.089527 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.089500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:07.125950 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.125894 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:07.252456 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.251968 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:07.255239 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:07.255191 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e011ec_a710_4936_a1b0_97b81ee7ab59.slice/crio-047c4ef003a0d6ed78d76ffd04eeee4e232a625ca05387da91f7daf81d9f2dea WatchSource:0}: Error finding container 047c4ef003a0d6ed78d76ffd04eeee4e232a625ca05387da91f7daf81d9f2dea: Status 404 returned error can't find the container with id 047c4ef003a0d6ed78d76ffd04eeee4e232a625ca05387da91f7daf81d9f2dea Jul 24 22:44:07.304777 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.304753 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79678c6f7-hl89h"] Jul 24 22:44:07.306215 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:07.306187 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732e595b_3bfe_414e_9381_83a63f8c4d5c.slice/crio-1e254367b1424cce16aff672f4cca7af12635e1c496c0d3c99dd8c387ecaa781 WatchSource:0}: Error finding container 1e254367b1424cce16aff672f4cca7af12635e1c496c0d3c99dd8c387ecaa781: Status 404 returned error can't find the container with id 1e254367b1424cce16aff672f4cca7af12635e1c496c0d3c99dd8c387ecaa781 Jul 24 22:44:07.778251 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.778191 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79678c6f7-hl89h" event={"ID":"732e595b-3bfe-414e-9381-83a63f8c4d5c","Type":"ContainerStarted","Data":"1e254367b1424cce16aff672f4cca7af12635e1c496c0d3c99dd8c387ecaa781"} Jul 24 22:44:07.780313 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:07.780280 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7875b74595-jgltg" event={"ID":"67e011ec-a710-4936-a1b0-97b81ee7ab59","Type":"ContainerStarted","Data":"047c4ef003a0d6ed78d76ffd04eeee4e232a625ca05387da91f7daf81d9f2dea"} Jul 24 22:44:10.802528 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.802491 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79678c6f7-hl89h" event={"ID":"732e595b-3bfe-414e-9381-83a63f8c4d5c","Type":"ContainerStarted","Data":"8593f877a4fdea8f531a73570fb60786f48f6efa45205cb76c262a28256e385e"} Jul 24 22:44:10.802982 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.802620 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:10.804069 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.804045 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7875b74595-jgltg" event={"ID":"67e011ec-a710-4936-a1b0-97b81ee7ab59","Type":"ContainerStarted","Data":"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a"} Jul 24 22:44:10.804197 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.804180 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:10.821511 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.821464 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79678c6f7-hl89h" podStartSLOduration=2.39056435 podStartE2EDuration="4.821451687s" totalImagesPullingTime="2.430887337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:10.820722696 +0000 UTC m=+514.027961506" watchObservedRunningTime="2026-07-24 22:44:10.821451687 +0000 UTC m=+514.028690495" Jul 24 22:44:10.838288 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:10.838248 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7875b74595-jgltg" podStartSLOduration=2.344264494 podStartE2EDuration="4.838221769s" totalImagesPullingTime="2.493957275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:10.836756783 +0000 UTC m=+514.043995592" watchObservedRunningTime="2026-07-24 22:44:10.838221769 +0000 UTC m=+514.045460574" Jul 24 22:44:21.811687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:21.811654 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:21.812118 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:21.812045 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79678c6f7-hl89h" Jul 24 22:44:21.875474 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:21.875443 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:21.875679 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:21.875653 2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7875b74595-jgltg" podUID="67e011ec-a710-4936-a1b0-97b81ee7ab59" containerName="manager" containerID="cri-o://52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a" gracePeriod=30 Jul 24 22:44:22.128131 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.128108 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:22.305144 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.305118 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcgkw\" (UniqueName: \"kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw\") pod \"67e011ec-a710-4936-a1b0-97b81ee7ab59\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " Jul 24 22:44:22.305326 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.305173 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert\") pod \"67e011ec-a710-4936-a1b0-97b81ee7ab59\" (UID: \"67e011ec-a710-4936-a1b0-97b81ee7ab59\") " Jul 24 22:44:22.307703 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.307676 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "67e011ec-a710-4936-a1b0-97b81ee7ab59" (UID: "67e011ec-a710-4936-a1b0-97b81ee7ab59"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:44:22.307806 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.307744 2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw" (OuterVolumeSpecName: "kube-api-access-zcgkw") pod "67e011ec-a710-4936-a1b0-97b81ee7ab59" (UID: "67e011ec-a710-4936-a1b0-97b81ee7ab59"). InnerVolumeSpecName "kube-api-access-zcgkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:22.406760 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.406703 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67e011ec-a710-4936-a1b0-97b81ee7ab59-webhook-cert\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:44:22.406760 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.406725 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcgkw\" (UniqueName: \"kubernetes.io/projected/67e011ec-a710-4936-a1b0-97b81ee7ab59-kube-api-access-zcgkw\") on node \"ip-10-0-130-123.ec2.internal\" DevicePath \"\"" Jul 24 22:44:22.866712 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.866681 2559 generic.go:356] "Generic (PLEG): container finished" podID="67e011ec-a710-4936-a1b0-97b81ee7ab59" containerID="52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a" exitCode=0 Jul 24 22:44:22.867145 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.866723 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7875b74595-jgltg" Jul 24 22:44:22.867145 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.866812 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7875b74595-jgltg" event={"ID":"67e011ec-a710-4936-a1b0-97b81ee7ab59","Type":"ContainerDied","Data":"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a"} Jul 24 22:44:22.867145 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.866869 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7875b74595-jgltg" event={"ID":"67e011ec-a710-4936-a1b0-97b81ee7ab59","Type":"ContainerDied","Data":"047c4ef003a0d6ed78d76ffd04eeee4e232a625ca05387da91f7daf81d9f2dea"} Jul 24 22:44:22.867145 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.866892 2559 scope.go:117] "RemoveContainer" containerID="52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a" Jul 24 22:44:22.876867 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.876717 2559 scope.go:117] "RemoveContainer" containerID="52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a" Jul 24 22:44:22.876966 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:22.876946 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a\": container with ID starting with 52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a not found: ID does not exist" containerID="52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a" Jul 24 22:44:22.877009 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.876976 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a"} err="failed to get container status \"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a\": rpc error: code = NotFound desc = could not find container \"52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a\": container with ID starting with 52dcc8a9fa0b192d283784243016da69f7224d2864e9eb6e538e7b7819b5761a not found: ID does not exist" Jul 24 22:44:22.889795 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.889775 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:22.893258 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:22.893239 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7875b74595-jgltg"] Jul 24 22:44:23.214119 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:23.214050 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e011ec-a710-4936-a1b0-97b81ee7ab59" path="/var/lib/kubelet/pods/67e011ec-a710-4936-a1b0-97b81ee7ab59/volumes" Jul 24 22:44:41.377161 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.377127 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76d9db859c-grnzk"] Jul 24 22:44:41.378489 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.378464 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e011ec-a710-4936-a1b0-97b81ee7ab59" containerName="manager" Jul 24 22:44:41.378640 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.378628 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e011ec-a710-4936-a1b0-97b81ee7ab59" containerName="manager" Jul 24 22:44:41.378909 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.378896 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e011ec-a710-4936-a1b0-97b81ee7ab59" containerName="manager" Jul 24 22:44:41.384462 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.384436 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.387595 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.387575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 24 22:44:41.387714 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.387703 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g5gbf\"" Jul 24 22:44:41.395374 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.395350 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d9db859c-grnzk"] Jul 24 22:44:41.471401 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.471374 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1344c919-8a69-459a-b7dd-8ffdfe182030-openshift-service-ca\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.471526 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.471402 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt9fl\" (UniqueName: \"kubernetes.io/projected/1344c919-8a69-459a-b7dd-8ffdfe182030-kube-api-access-mt9fl\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.471526 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.471479 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.572593 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.572565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.572700 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.572638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1344c919-8a69-459a-b7dd-8ffdfe182030-openshift-service-ca\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.572700 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.572669 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt9fl\" (UniqueName: \"kubernetes.io/projected/1344c919-8a69-459a-b7dd-8ffdfe182030-kube-api-access-mt9fl\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.572793 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:41.572709 2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 24 22:44:41.572793 ip-10-0-130-123 kubenswrapper[2559]: E0724 22:44:41.572785 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls podName:1344c919-8a69-459a-b7dd-8ffdfe182030 nodeName:}" failed. No retries permitted until 2026-07-24 22:44:42.072762499 +0000 UTC m=+545.280001286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls") pod "maas-api-76d9db859c-grnzk" (UID: "1344c919-8a69-459a-b7dd-8ffdfe182030") : secret "maas-api-serving-cert" not found Jul 24 22:44:41.573419 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.573398 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1344c919-8a69-459a-b7dd-8ffdfe182030-openshift-service-ca\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.583776 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.583752 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt9fl\" (UniqueName: \"kubernetes.io/projected/1344c919-8a69-459a-b7dd-8ffdfe182030-kube-api-access-mt9fl\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:41.924610 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.924579 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dc96x"] Jul 24 22:44:41.928215 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.928195 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:41.931383 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.931361 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 22:44:41.931497 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.931458 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fk77t\"" Jul 24 22:44:41.938660 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.938636 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dc96x"] Jul 24 22:44:41.975861 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.975826 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p84gc\" (UniqueName: \"kubernetes.io/projected/36352947-7bf9-4107-91d0-c8360aa36a46-kube-api-access-p84gc\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:41.976027 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.975868 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36352947-7bf9-4107-91d0-c8360aa36a46-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:41.992687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.992663 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bzrct"] Jul 24 22:44:41.996119 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:41.996104 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.006984 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.006962 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bzrct"] Jul 24 22:44:42.076555 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.076523 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gc9h\" (UniqueName: \"kubernetes.io/projected/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-kube-api-access-7gc9h\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.076725 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.076559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p84gc\" (UniqueName: \"kubernetes.io/projected/36352947-7bf9-4107-91d0-c8360aa36a46-kube-api-access-p84gc\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:42.076725 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.076581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36352947-7bf9-4107-91d0-c8360aa36a46-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:42.076725 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.076606 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:42.076725 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.076664 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.077474 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.077453 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36352947-7bf9-4107-91d0-c8360aa36a46-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:42.079249 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.079197 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1344c919-8a69-459a-b7dd-8ffdfe182030-maas-api-tls\") pod \"maas-api-76d9db859c-grnzk\" (UID: \"1344c919-8a69-459a-b7dd-8ffdfe182030\") " pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:42.084942 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.084917 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p84gc\" (UniqueName: \"kubernetes.io/projected/36352947-7bf9-4107-91d0-c8360aa36a46-kube-api-access-p84gc\") pod \"payload-processing-5bf47cd988-dc96x\" (UID: \"36352947-7bf9-4107-91d0-c8360aa36a46\") " pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:42.177518 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.177446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.177632 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.177535 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gc9h\" (UniqueName: \"kubernetes.io/projected/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-kube-api-access-7gc9h\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.178072 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.178054 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.185689 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.185668 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gc9h\" (UniqueName: \"kubernetes.io/projected/c41bad9e-2a6d-4301-b202-ac5ded4a0bc1-kube-api-access-7gc9h\") pod \"payload-pre-processing-7677b774bb-bzrct\" (UID: \"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.239641 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.239617 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:42.296165 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.296109 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:42.306818 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.306562 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:42.392507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.392127 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dc96x"] Jul 24 22:44:42.396525 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:42.396432 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36352947_7bf9_4107_91d0_c8360aa36a46.slice/crio-085987578a4f8bfd4f66f16fa47f06d1d16b1c26a1e1774c5b47e5c0c27ecb9d WatchSource:0}: Error finding container 085987578a4f8bfd4f66f16fa47f06d1d16b1c26a1e1774c5b47e5c0c27ecb9d: Status 404 returned error can't find the container with id 085987578a4f8bfd4f66f16fa47f06d1d16b1c26a1e1774c5b47e5c0c27ecb9d Jul 24 22:44:42.477992 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.477967 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d9db859c-grnzk"] Jul 24 22:44:42.479246 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:42.479205 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1344c919_8a69_459a_b7dd_8ffdfe182030.slice/crio-b9b994212d94c5703dc91e12318347f181bf5072e2cb169661cc373cf867f73d WatchSource:0}: Error finding container b9b994212d94c5703dc91e12318347f181bf5072e2cb169661cc373cf867f73d: Status 404 returned error can't find the container with id b9b994212d94c5703dc91e12318347f181bf5072e2cb169661cc373cf867f73d Jul 24 22:44:42.496833 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.496810 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-bzrct"] Jul 24 22:44:42.498137 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:44:42.498118 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41bad9e_2a6d_4301_b202_ac5ded4a0bc1.slice/crio-28f1c4f86f655b290a2d86b9a53165465bbca28c3433f551d6ba24817770b524 WatchSource:0}: Error finding container 28f1c4f86f655b290a2d86b9a53165465bbca28c3433f551d6ba24817770b524: Status 404 returned error can't find the container with id 28f1c4f86f655b290a2d86b9a53165465bbca28c3433f551d6ba24817770b524 Jul 24 22:44:42.968586 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.968557 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" event={"ID":"1344c919-8a69-459a-b7dd-8ffdfe182030","Type":"ContainerStarted","Data":"b9b994212d94c5703dc91e12318347f181bf5072e2cb169661cc373cf867f73d"} Jul 24 22:44:42.969968 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.969943 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" event={"ID":"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1","Type":"ContainerStarted","Data":"28f1c4f86f655b290a2d86b9a53165465bbca28c3433f551d6ba24817770b524"} Jul 24 22:44:42.971168 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:42.971140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" event={"ID":"36352947-7bf9-4107-91d0-c8360aa36a46","Type":"ContainerStarted","Data":"085987578a4f8bfd4f66f16fa47f06d1d16b1c26a1e1774c5b47e5c0c27ecb9d"} Jul 24 22:44:45.989109 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.989080 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" event={"ID":"c41bad9e-2a6d-4301-b202-ac5ded4a0bc1","Type":"ContainerStarted","Data":"973775b64a4e9a7f505f225dc97c3e52ae27ffdb6dde2135104fd8d5d71dbb58"} Jul 24 22:44:45.989519 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.989142 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:44:45.990746 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.990722 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" event={"ID":"36352947-7bf9-4107-91d0-c8360aa36a46","Type":"ContainerStarted","Data":"16a6c5c898702baae5b8502bf3a2b8002c6e7d2da0fcd3595b3e872a34d5fa37"} Jul 24 22:44:45.990883 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.990772 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:45.992092 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.992071 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" event={"ID":"1344c919-8a69-459a-b7dd-8ffdfe182030","Type":"ContainerStarted","Data":"6a169e3177dd2b13a96c0521bf6a271c805394b0fe57cdc48c4dc34f4220bf6b"} Jul 24 22:44:45.992200 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:45.992185 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:46.004893 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:46.004849 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" podStartSLOduration=2.229653942 podStartE2EDuration="5.004835729s" totalImagesPullingTime="2.775181787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:46.004129831 +0000 UTC m=+549.211368640" watchObservedRunningTime="2026-07-24 22:44:46.004835729 +0000 UTC m=+549.212074515" Jul 24 22:44:46.022341 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:46.022304 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" podStartSLOduration=2.290389094 podStartE2EDuration="5.022293229s" totalImagesPullingTime="2.731904135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:46.021631403 +0000 UTC m=+549.228870213" watchObservedRunningTime="2026-07-24 22:44:46.022293229 +0000 UTC m=+549.229532035" Jul 24 22:44:46.037413 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:46.037371 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" podStartSLOduration=2.154250313 podStartE2EDuration="5.037354686s" totalImagesPullingTime="2.883104373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:46.036274353 +0000 UTC m=+549.243513183" watchObservedRunningTime="2026-07-24 22:44:46.037354686 +0000 UTC m=+549.244593495" Jul 24 22:44:52.002801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:52.002773 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76d9db859c-grnzk" Jul 24 22:44:55.954852 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:55.954823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69bc55c65d-zs5kt_7a688ba1-11bb-48aa-abda-7f8101dee067/authorino/0.log" Jul 24 22:44:56.999625 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:56.999596 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-dc96x" Jul 24 22:44:57.000083 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:44:56.999771 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-bzrct" Jul 24 22:45:01.279858 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.279824 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-q82fl_23f06a1d-fea8-4cc8-8cd3-97c607db5d78/manager/0.log" Jul 24 22:45:01.517523 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.517496 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79678c6f7-hl89h_732e595b-3bfe-414e-9381-83a63f8c4d5c/manager/0.log" Jul 24 22:45:01.630438 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.630411 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z8ttg_db6acc5f-f292-4d42-ab69-3a7719de8da5/server/0.log" Jul 24 22:45:01.746914 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.746889 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hrtvn_2b1a43b1-463c-468d-b62c-a1bdfdd93aef/manager/0.log" Jul 24 22:45:01.871558 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.871528 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d7b955f-6z9qz_f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf/manager/0.log" Jul 24 22:45:01.884508 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:01.884450 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d7b955f-6z9qz_f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf/copy-manifests/0.log" Jul 24 22:45:02.486739 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:02.486712 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d9db859c-grnzk_1344c919-8a69-459a-b7dd-8ffdfe182030/maas-api/0.log" Jul 24 22:45:02.708642 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:02.708614 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-m9v72_28bdfb18-1435-4308-889e-1f60688bdd3d/postgres/0.log" Jul 24 22:45:03.353064 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.353033 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx"] Jul 24 22:45:03.358503 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.358471 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.362023 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.361997 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hlc8h\"" Jul 24 22:45:03.362496 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.362475 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 22:45:03.363475 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.363455 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 22:45:03.363595 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.363458 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 22:45:03.370423 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.370402 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx"] Jul 24 22:45:03.459810 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.459775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.459987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.459845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.459987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.459936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.459987 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.459963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.460105 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.460027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8114173-0d26-42f5-bc15-44accaf0119f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.460105 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.460048 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-847zq\" (UniqueName: \"kubernetes.io/projected/e8114173-0d26-42f5-bc15-44accaf0119f-kube-api-access-847zq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561420 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561434 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561454 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561494 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8114173-0d26-42f5-bc15-44accaf0119f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561510 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-847zq\" (UniqueName: \"kubernetes.io/projected/e8114173-0d26-42f5-bc15-44accaf0119f-kube-api-access-847zq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.561824 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561531 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.562113 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561952 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.562113 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561949 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.562113 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.561995 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.563801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.563777 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8114173-0d26-42f5-bc15-44accaf0119f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.564051 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.564036 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8114173-0d26-42f5-bc15-44accaf0119f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.575749 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.575722 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-847zq\" (UniqueName: \"kubernetes.io/projected/e8114173-0d26-42f5-bc15-44accaf0119f-kube-api-access-847zq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx\" (UID: \"e8114173-0d26-42f5-bc15-44accaf0119f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.672612 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.672531 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:03.826120 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:03.826094 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx"] Jul 24 22:45:03.827990 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:45:03.827962 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8114173_0d26_42f5_bc15_44accaf0119f.slice/crio-065075cd9ea600ff459538a40dfce4d23691ce1b92d31f467a80f001ff268b8a WatchSource:0}: Error finding container 065075cd9ea600ff459538a40dfce4d23691ce1b92d31f467a80f001ff268b8a: Status 404 returned error can't find the container with id 065075cd9ea600ff459538a40dfce4d23691ce1b92d31f467a80f001ff268b8a Jul 24 22:45:04.062751 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.062671 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69bc55c65d-zs5kt_7a688ba1-11bb-48aa-abda-7f8101dee067/authorino/0.log" Jul 24 22:45:04.089883 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.089851 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" event={"ID":"e8114173-0d26-42f5-bc15-44accaf0119f","Type":"ContainerStarted","Data":"065075cd9ea600ff459538a40dfce4d23691ce1b92d31f467a80f001ff268b8a"} Jul 24 22:45:04.185839 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.185807 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-srzdk_d0edc10f-ba97-4554-ba7a-87dd77e36654/manager/0.log" Jul 24 22:45:04.315333 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.315253 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-84c6b_cbd2bd20-e9ce-4d7e-897c-f23122eb370a/manager/0.log" Jul 24 22:45:04.425068 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.425039 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9tl2p_8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07/kuadrant-console-plugin/0.log" Jul 24 22:45:04.537623 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.537595 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-47cw7_3dc03afe-c0a4-410f-a5f8-8697816c3b49/registry-server/0.log" Jul 24 22:45:04.904041 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:04.904018 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-pnhtk_27910791-96a1-44b8-a523-21c10f56c6fa/manager/0.log" Jul 24 22:45:05.514850 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:05.514818 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f7697f5d-5whnh_704d40dc-469a-43bd-817c-4125ef46a881/kube-auth-proxy/0.log" Jul 24 22:45:05.865467 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:05.865435 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bzrct_c41bad9e-2a6d-4301-b202-ac5ded4a0bc1/payload-pre-processing/0.log" Jul 24 22:45:05.985430 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:05.985390 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dc96x_36352947-7bf9-4107-91d0-c8360aa36a46/payload-processing/0.log" Jul 24 22:45:06.102255 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:06.102209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" event={"ID":"e8114173-0d26-42f5-bc15-44accaf0119f","Type":"ContainerStarted","Data":"4b824d57214fd183725dd46e6c476f77faf5330c99d6ba41dd2eb4eabb958d23"} Jul 24 22:45:06.105626 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:06.105602 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68bcdfbddf-x2qc8_2edd2a28-e797-4584-a305-a81fae76f1d0/router/0.log" Jul 24 22:45:06.121547 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:06.121471 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" podStartSLOduration=0.986837782 podStartE2EDuration="3.121455985s" totalImagesPullingTime="2.134618203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:45:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:06.12039909 +0000 UTC m=+569.327637931" watchObservedRunningTime="2026-07-24 22:45:06.121455985 +0000 UTC m=+569.328694791" Jul 24 22:45:06.445404 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:06.445334 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx_e8114173-0d26-42f5-bc15-44accaf0119f/main/0.log" Jul 24 22:45:13.672862 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:13.672825 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:13.673494 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:13.672881 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:13.685169 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:13.685146 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:14.152877 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:14.152848 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-l4xhx" Jul 24 22:45:15.354481 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:15.354451 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mfq5d_8d839f85-78a3-4588-b9d7-b7de549c7d3d/global-pull-secret-syncer/0.log" Jul 24 22:45:15.497661 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:15.497635 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-shbtf_8cde2a49-cf6d-440a-9b1b-98f9539f7ef4/konnectivity-agent/0.log" Jul 24 22:45:15.516482 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:15.516444 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-123.ec2.internal_542ff756433833167701e75caa9f94dd/haproxy/0.log" Jul 24 22:45:20.237655 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.237627 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-69bc55c65d-zs5kt_7a688ba1-11bb-48aa-abda-7f8101dee067/authorino/0.log" Jul 24 22:45:20.278529 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.278486 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-srzdk_d0edc10f-ba97-4554-ba7a-87dd77e36654/manager/0.log" Jul 24 22:45:20.316183 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.316164 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-84c6b_cbd2bd20-e9ce-4d7e-897c-f23122eb370a/manager/0.log" Jul 24 22:45:20.341994 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.341934 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9tl2p_8ecae87b-9caa-4ec6-9b10-f2d74ffe4e07/kuadrant-console-plugin/0.log" Jul 24 22:45:20.368532 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.368512 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-47cw7_3dc03afe-c0a4-410f-a5f8-8697816c3b49/registry-server/0.log" Jul 24 22:45:20.468774 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:20.468755 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-pnhtk_27910791-96a1-44b8-a523-21c10f56c6fa/manager/0.log" Jul 24 22:45:22.320556 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:22.320527 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-6vdvk_ca775d01-4dc6-409e-bd30-dc3337e7f028/cluster-monitoring-operator/0.log" Jul 24 22:45:22.654376 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:22.654349 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prhdt_f2b2a315-0a55-4017-8137-416a517c4463/node-exporter/0.log" Jul 24 22:45:22.686553 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:22.686530 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prhdt_f2b2a315-0a55-4017-8137-416a517c4463/kube-rbac-proxy/0.log" Jul 24 22:45:22.709323 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:22.709301 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prhdt_f2b2a315-0a55-4017-8137-416a517c4463/init-textfile/0.log" Jul 24 22:45:24.808465 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:24.808439 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-p79sp_1d378d4b-73a5-4839-8a5e-febd01167e24/networking-console-plugin/0.log" Jul 24 22:45:25.469039 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.469013 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:45:25.478792 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.474941 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/2.log" Jul 24 22:45:25.891577 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.891548 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2"] Jul 24 22:45:25.898550 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.898533 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:25.901884 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.901859 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvv68\"/\"kube-root-ca.crt\"" Jul 24 22:45:25.902019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.901894 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvv68\"/\"openshift-service-ca.crt\"" Jul 24 22:45:25.902019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.901894 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kvv68\"/\"default-dockercfg-jjj5p\"" Jul 24 22:45:25.903080 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.903059 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2"] Jul 24 22:45:25.943763 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.943738 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4jx6\" (UniqueName: \"kubernetes.io/projected/bce1e8e9-7360-40e9-a4d0-2dd237159c99-kube-api-access-n4jx6\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:25.943921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.943798 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-podres\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:25.943921 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.943900 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-proc\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:25.944048 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.943940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-sys\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:25.944048 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:25.944006 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-lib-modules\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.044950 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.044922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4jx6\" (UniqueName: \"kubernetes.io/projected/bce1e8e9-7360-40e9-a4d0-2dd237159c99-kube-api-access-n4jx6\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045114 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.044984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-podres\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045114 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045036 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-proc\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045114 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045070 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-sys\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045114 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-lib-modules\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045153 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-proc\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045198 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-podres\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045211 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-sys\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.045329 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.045256 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bce1e8e9-7360-40e9-a4d0-2dd237159c99-lib-modules\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.053202 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.053180 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4jx6\" (UniqueName: \"kubernetes.io/projected/bce1e8e9-7360-40e9-a4d0-2dd237159c99-kube-api-access-n4jx6\") pod \"perf-node-gather-daemonset-kxlv2\" (UID: \"bce1e8e9-7360-40e9-a4d0-2dd237159c99\") " pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.064065 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.064048 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-cwnrf_435b83b2-108b-4edb-becb-c7c3702db1d2/download-server/0.log" Jul 24 22:45:26.209138 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.209071 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:26.342808 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.342780 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2"] Jul 24 22:45:26.345337 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:45:26.345315 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbce1e8e9_7360_40e9_a4d0_2dd237159c99.slice/crio-3407be6769eda73ad802cb19c2fd19fc98e736518cf2e4c34d6d209a72ac79dc WatchSource:0}: Error finding container 3407be6769eda73ad802cb19c2fd19fc98e736518cf2e4c34d6d209a72ac79dc: Status 404 returned error can't find the container with id 3407be6769eda73ad802cb19c2fd19fc98e736518cf2e4c34d6d209a72ac79dc Jul 24 22:45:26.666708 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:26.666684 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-swsns_a5a49982-99c8-43df-82be-1d74ecf06753/volume-data-source-validator/0.log" Jul 24 22:45:27.215263 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.215209 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" event={"ID":"bce1e8e9-7360-40e9-a4d0-2dd237159c99","Type":"ContainerStarted","Data":"a6223335980927d338e9f7e508ebc8fcb2041e9536e648561f4443b4c5ca5211"} Jul 24 22:45:27.215624 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.215272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" event={"ID":"bce1e8e9-7360-40e9-a4d0-2dd237159c99","Type":"ContainerStarted","Data":"3407be6769eda73ad802cb19c2fd19fc98e736518cf2e4c34d6d209a72ac79dc"} Jul 24 22:45:27.215624 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.215330 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:27.233539 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.233488 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" podStartSLOduration=2.233476123 podStartE2EDuration="2.233476123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:27.231168976 +0000 UTC m=+590.438407804" watchObservedRunningTime="2026-07-24 22:45:27.233476123 +0000 UTC m=+590.440714988" Jul 24 22:45:27.745075 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.745045 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mdjzv_74e5ce1f-4f12-49d5-9d35-542cbeb3bc34/dns/0.log" Jul 24 22:45:27.768306 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.768285 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mdjzv_74e5ce1f-4f12-49d5-9d35-542cbeb3bc34/kube-rbac-proxy/0.log" Jul 24 22:45:27.887919 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:27.887900 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kv4dv_5f4f9dfe-beef-4b50-a0e3-62f8d61e0178/dns-node-resolver/0.log" Jul 24 22:45:28.474620 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:28.474596 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5wqk_b0936fc0-4e09-4ec8-9759-183233c4020f/node-ca/0.log" Jul 24 22:45:29.494627 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:29.494596 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f7697f5d-5whnh_704d40dc-469a-43bd-817c-4125ef46a881/kube-auth-proxy/0.log" Jul 24 22:45:29.572707 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:29.572679 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bzrct_c41bad9e-2a6d-4301-b202-ac5ded4a0bc1/payload-pre-processing/0.log" Jul 24 22:45:29.598652 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:29.598628 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dc96x_36352947-7bf9-4107-91d0-c8360aa36a46/payload-processing/0.log" Jul 24 22:45:29.624178 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:29.624153 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68bcdfbddf-x2qc8_2edd2a28-e797-4584-a305-a81fae76f1d0/router/0.log" Jul 24 22:45:30.245781 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.245737 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h2chv_cdb02f3c-8fca-4daa-b79b-63e8456559ea/serve-healthcheck-canary/0.log" Jul 24 22:45:30.771818 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.771789 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-qdl87_40f6c541-38fd-4e28-8d80-80f2fa2c7cb3/insights-operator/1.log" Jul 24 22:45:30.777431 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.777401 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-qdl87_40f6c541-38fd-4e28-8d80-80f2fa2c7cb3/insights-operator/0.log" Jul 24 22:45:30.797177 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.797160 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmgbj_788fe0e1-ae93-41a9-a3be-2890ca680fd0/kube-rbac-proxy/0.log" Jul 24 22:45:30.818687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.818669 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmgbj_788fe0e1-ae93-41a9-a3be-2890ca680fd0/exporter/0.log" Jul 24 22:45:30.840449 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:30.840429 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bmgbj_788fe0e1-ae93-41a9-a3be-2890ca680fd0/extractor/0.log" Jul 24 22:45:33.089658 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.089610 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-q82fl_23f06a1d-fea8-4cc8-8cd3-97c607db5d78/manager/0.log" Jul 24 22:45:33.140333 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.140313 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79678c6f7-hl89h_732e595b-3bfe-414e-9381-83a63f8c4d5c/manager/0.log" Jul 24 22:45:33.162429 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.162409 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z8ttg_db6acc5f-f292-4d42-ab69-3a7719de8da5/server/0.log" Jul 24 22:45:33.209541 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.209514 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-hrtvn_2b1a43b1-463c-468d-b62c-a1bdfdd93aef/manager/0.log" Jul 24 22:45:33.231025 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.231005 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kvv68/perf-node-gather-daemonset-kxlv2" Jul 24 22:45:33.249946 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.249926 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d7b955f-6z9qz_f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf/manager/0.log" Jul 24 22:45:33.266517 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:33.266499 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-665d7b955f-6z9qz_f3e2a12d-0cc7-48d9-b23b-2e0ca4560bdf/copy-manifests/0.log" Jul 24 22:45:35.901093 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:35.901068 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f7697f5d-5whnh_704d40dc-469a-43bd-817c-4125ef46a881/kube-auth-proxy/0.log" Jul 24 22:45:35.951427 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:35.951406 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bzrct_c41bad9e-2a6d-4301-b202-ac5ded4a0bc1/payload-pre-processing/0.log" Jul 24 22:45:35.965215 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:35.965192 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dc96x_36352947-7bf9-4107-91d0-c8360aa36a46/payload-processing/0.log" Jul 24 22:45:35.983989 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:35.983972 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68bcdfbddf-x2qc8_2edd2a28-e797-4584-a305-a81fae76f1d0/router/0.log" Jul 24 22:45:37.092372 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.092344 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65f7697f5d-5whnh_704d40dc-469a-43bd-817c-4125ef46a881/kube-auth-proxy/0.log" Jul 24 22:45:37.146755 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.146683 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-bzrct_c41bad9e-2a6d-4301-b202-ac5ded4a0bc1/payload-pre-processing/0.log" Jul 24 22:45:37.152620 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.152599 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:45:37.154973 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.154952 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-bdgq6_22f63a83-bc41-41ae-b1ca-6d860473d275/console-operator/1.log" Jul 24 22:45:37.155484 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.155468 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:45:37.157867 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.157849 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lplhc_4f947503-67bd-4d12-88c7-9bf939ed72b6/ovn-acl-logging/0.log" Jul 24 22:45:37.162450 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.162429 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dc96x_36352947-7bf9-4107-91d0-c8360aa36a46/payload-processing/0.log" Jul 24 22:45:37.183809 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:37.183770 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-68bcdfbddf-x2qc8_2edd2a28-e797-4584-a305-a81fae76f1d0/router/0.log" Jul 24 22:45:38.671982 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.671952 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln"] Jul 24 22:45:38.686352 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.686328 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln"] Jul 24 22:45:38.686505 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.686417 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.689567 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.689544 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 24 22:45:38.733890 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.733861 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.734019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.733900 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/528362fa-3eb9-4a38-a4f4-839d4e85da01-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.734019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.733932 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpb9g\" (UniqueName: \"kubernetes.io/projected/528362fa-3eb9-4a38-a4f4-839d4e85da01-kube-api-access-zpb9g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.734019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.733962 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.734019 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.733994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.734268 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.734124 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.834687 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.834656 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.834801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.834691 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/528362fa-3eb9-4a38-a4f4-839d4e85da01-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.834801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.834713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpb9g\" (UniqueName: \"kubernetes.io/projected/528362fa-3eb9-4a38-a4f4-839d4e85da01-kube-api-access-zpb9g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.834801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.834731 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.834801 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.834752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.835093 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.835019 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.835093 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.835073 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.835206 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.835092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.835206 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.835108 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.837522 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.837500 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/528362fa-3eb9-4a38-a4f4-839d4e85da01-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.837611 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.837582 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/528362fa-3eb9-4a38-a4f4-839d4e85da01-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.843693 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.843672 2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpb9g\" (UniqueName: \"kubernetes.io/projected/528362fa-3eb9-4a38-a4f4-839d4e85da01-kube-api-access-zpb9g\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g66ln\" (UID: \"528362fa-3eb9-4a38-a4f4-839d4e85da01\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:38.999349 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:38.999281 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" Jul 24 22:45:39.147638 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:39.147615 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln"] Jul 24 22:45:39.149494 ip-10-0-130-123 kubenswrapper[2559]: W0724 22:45:39.149455 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod528362fa_3eb9_4a38_a4f4_839d4e85da01.slice/crio-c79e180a1f1a7b1c9f40e9ea1f004ce6ae5b5e7fcfbcf916933151a33c1ac06f WatchSource:0}: Error finding container c79e180a1f1a7b1c9f40e9ea1f004ce6ae5b5e7fcfbcf916933151a33c1ac06f: Status 404 returned error can't find the container with id c79e180a1f1a7b1c9f40e9ea1f004ce6ae5b5e7fcfbcf916933151a33c1ac06f Jul 24 22:45:39.274564 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:39.274504 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" event={"ID":"528362fa-3eb9-4a38-a4f4-839d4e85da01","Type":"ContainerStarted","Data":"c79e180a1f1a7b1c9f40e9ea1f004ce6ae5b5e7fcfbcf916933151a33c1ac06f"} Jul 24 22:45:39.967327 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:39.967296 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fmnxq_2b836276-d3af-4232-b7c2-d0c028c64e7e/migrator/0.log" Jul 24 22:45:39.992507 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:39.992484 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fmnxq_2b836276-d3af-4232-b7c2-d0c028c64e7e/graceful-termination/0.log" Jul 24 22:45:40.281410 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:40.281323 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" event={"ID":"528362fa-3eb9-4a38-a4f4-839d4e85da01","Type":"ContainerStarted","Data":"a87f2d0c6a2918e158940692636236d79849c96a3c4e241ad0d2605739cddb09"} Jul 24 22:45:40.300287 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:40.300213 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g66ln" podStartSLOduration=2.1191557 podStartE2EDuration="2.300197109s" totalImagesPullingTime="181.041409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:40.298278291 +0000 UTC m=+603.505517105" watchObservedRunningTime="2026-07-24 22:45:40.300197109 +0000 UTC m=+603.507435914" Jul 24 22:45:40.401381 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:40.401346 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-npd46_69dea1e1-26cb-408d-b7db-23f1aeed1250/kube-storage-version-migrator-operator/0.log" Jul 24 22:45:40.402454 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:40.402425 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-npd46_69dea1e1-26cb-408d-b7db-23f1aeed1250/kube-storage-version-migrator-operator/1.log" Jul 24 22:45:41.375660 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.375633 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/kube-multus-additional-cni-plugins/0.log" Jul 24 22:45:41.402723 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.402702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/egress-router-binary-copy/0.log" Jul 24 22:45:41.426885 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.426865 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/cni-plugins/0.log" Jul 24 22:45:41.450392 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.450376 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/bond-cni-plugin/0.log" Jul 24 22:45:41.472692 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.472670 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/routeoverride-cni/0.log" Jul 24 22:45:41.496698 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.496679 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/whereabouts-cni-bincopy/0.log" Jul 24 22:45:41.520407 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.520391 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jhbt8_acb8d463-3918-4645-afd2-26855ea8f5dc/whereabouts-cni/0.log" Jul 24 22:45:41.894359 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:41.894338 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cgnnz_5e4b8fc2-3980-4526-8d47-c519c7920505/kube-multus/0.log" Jul 24 22:45:42.014597 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:42.014569 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bvd5v_ad0f0815-c320-4476-8580-cc6f0504457f/network-metrics-daemon/0.log" Jul 24 22:45:42.034037 ip-10-0-130-123 kubenswrapper[2559]: I0724 22:45:42.034017 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bvd5v_ad0f0815-c320-4476-8580-cc6f0504457f/kube-rbac-proxy/0.log"