Jul 01 18:37:19.752143 ip-10-0-139-178 systemd[1]: Starting Kubernetes Kubelet... Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.142126 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150070 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150082 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150086 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150090 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150093 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150096 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150099 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.163572 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150102 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150105 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150108 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150110 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150113 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150119 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150122 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150125 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150128 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150131 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150134 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150136 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150141 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150145 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150148 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150150 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150153 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150156 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150158 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.166130 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150161 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150164 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150167 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150169 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150172 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150175 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150177 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150181 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150184 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150187 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150189 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150192 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150194 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150198 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150202 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150205 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150207 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150210 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150213 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150216 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.167763 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150218 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150221 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150224 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150227 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150229 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150232 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150234 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150237 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150240 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150243 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150246 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150248 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150251 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150253 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150256 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150258 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150261 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150263 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150266 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150270 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.172114 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150273 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150276 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150279 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150282 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150284 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150287 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150289 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150292 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150295 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150298 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150300 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150303 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150306 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150308 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150311 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150313 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150316 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150318 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150321 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150323 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.185377 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150326 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150329 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150332 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150334 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150337 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150340 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150342 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150345 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150347 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150353 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150355 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150358 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.150360 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151077 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151084 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151087 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151090 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151093 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151097 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151100 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.200526 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151103 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151106 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151109 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151112 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151114 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151117 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151119 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151122 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151124 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151127 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151130 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151132 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151135 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151138 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151142 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151146 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151148 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151151 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151153 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.301635 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151157 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.216616 ip-10-0-139-178 systemd[1]: Started Kubernetes Kubelet. Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151159 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151164 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151167 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151170 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151173 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151176 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151178 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151181 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151183 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151186 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151189 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151194 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151196 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151200 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151203 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151206 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151209 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151211 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151214 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.332370 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151216 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151219 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151223 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151226 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151229 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151232 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151235 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151238 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151241 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151243 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151246 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151249 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151251 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151254 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151258 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151260 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151262 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151265 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151269 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.494343 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151271 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151274 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151276 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151280 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151282 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151285 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151288 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151290 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151293 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151295 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151298 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151300 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151303 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151305 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151308 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151310 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151313 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151315 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151318 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151321 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.495028 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151325 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151328 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151330 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151333 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151336 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151338 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151341 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151344 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151347 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151349 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151352 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151354 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151357 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151359 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151717 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151721 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151724 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151727 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151730 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151734 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.495636 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151737 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151739 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151742 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151744 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151747 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151750 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151752 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151755 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151757 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151760 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151762 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151765 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151771 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151774 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151776 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151779 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151781 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151784 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151786 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151790 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.496290 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151792 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151795 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151797 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151800 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151803 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151805 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151808 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151811 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151813 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151816 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151818 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151822 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151824 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151827 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151829 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151832 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151835 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151837 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151840 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151842 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.497030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151844 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151847 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151849 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151852 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151870 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151873 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151876 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151878 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151881 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151883 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151886 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151890 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151893 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151895 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151900 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151904 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151908 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151911 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151914 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151918 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.497893 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151920 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151923 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151926 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151929 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151933 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151936 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151940 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151943 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151946 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151948 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151951 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151953 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151956 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151958 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151961 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151963 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151967 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151969 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151972 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.498726 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151974 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151977 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151980 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151982 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151986 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151988 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151991 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151993 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151996 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.151998 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152001 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152004 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152006 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152009 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152080 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152091 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152097 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152101 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152105 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152108 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152113 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 18:37:20.499357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152117 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152120 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152123 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152126 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152129 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152132 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152136 2554 flags.go:64] FLAG: --cgroup-root="" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152140 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152144 2554 flags.go:64] FLAG: --client-ca-file="" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152147 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152150 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152154 2554 flags.go:64] FLAG: --cluster-domain="" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152157 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152160 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152163 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152168 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152172 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152175 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152178 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152181 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152184 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152187 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152190 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152193 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 18:37:20.499987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152196 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152201 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152204 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152207 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152209 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152212 2554 flags.go:64] FLAG: --enable-server="true" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152216 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152220 2554 flags.go:64] FLAG: --event-burst="100" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152223 2554 flags.go:64] FLAG: --event-qps="50" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152226 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152229 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152231 2554 flags.go:64] FLAG: --eviction-hard="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152235 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152237 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152240 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152243 2554 flags.go:64] FLAG: --eviction-soft="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152247 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152250 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152253 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152256 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152259 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152261 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152264 2554 flags.go:64] FLAG: --feature-gates="" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152269 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152272 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 18:37:20.500672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152275 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152278 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152281 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152284 2554 flags.go:64] FLAG: --help="false" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152287 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-139-178.ec2.internal" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152290 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152293 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152296 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152299 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152302 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152305 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152308 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152311 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152313 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152317 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152320 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152322 2554 flags.go:64] FLAG: --kube-reserved="" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152325 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152328 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152331 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152334 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152337 2554 flags.go:64] FLAG: --lock-file="" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152339 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152343 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 18:37:20.501441 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152347 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152355 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152358 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152360 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152363 2554 flags.go:64] FLAG: --logging-format="text" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152366 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152370 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152373 2554 flags.go:64] FLAG: --manifest-url="" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152376 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152380 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152383 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152387 2554 flags.go:64] FLAG: --max-pods="110" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152390 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152393 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152396 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152399 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152402 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152405 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152407 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152414 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152418 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152420 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152423 2554 flags.go:64] FLAG: --pod-cidr="" Jul 01 18:37:20.502255 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152426 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152431 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152434 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152437 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152440 2554 flags.go:64] FLAG: --port="10250" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152443 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152446 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05a164119422e8a16" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152449 2554 flags.go:64] FLAG: --qos-reserved="" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152453 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152456 2554 flags.go:64] FLAG: --register-node="true" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152459 2554 flags.go:64] FLAG: --register-with-taints="" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152463 2554 flags.go:64] FLAG: --registry-burst="10" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152465 2554 flags.go:64] FLAG: --registry-qps="5" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152468 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152471 2554 flags.go:64] FLAG: --reserved-memory="" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152474 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152479 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152482 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152485 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152488 2554 flags.go:64] FLAG: --runonce="false" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152491 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152494 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152497 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152500 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152502 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152505 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 18:37:20.503107 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152508 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152511 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152514 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152517 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152520 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152523 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152525 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152529 2554 flags.go:64] FLAG: --system-cgroups="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152532 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152537 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152539 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152542 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152547 2554 flags.go:64] FLAG: --tls-min-version="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152549 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152553 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152556 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152559 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152562 2554 flags.go:64] FLAG: --v="2" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152565 2554 flags.go:64] FLAG: --version="false" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152569 2554 flags.go:64] FLAG: --vmodule="" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152572 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152576 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152690 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152694 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.504412 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152697 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152700 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152703 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152706 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152708 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152711 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152714 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152716 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152719 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152722 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152724 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152727 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152730 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152732 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152735 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152739 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152742 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152745 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152748 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152750 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.505440 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152753 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152756 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152760 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152762 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152765 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152768 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152770 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152773 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152775 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152778 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152782 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152784 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152787 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152790 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152792 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152795 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152797 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152800 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152802 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152805 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.506324 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152808 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152810 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152813 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152815 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152817 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152820 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152822 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152825 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152828 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152830 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152833 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152837 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152840 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152843 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152848 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152850 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152853 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152873 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152879 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.507263 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152884 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152888 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152892 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152896 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152899 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152902 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152905 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152908 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152911 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152914 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152916 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152919 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152922 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152925 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152928 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152930 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152933 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152935 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152938 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152940 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.508104 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152943 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152945 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152948 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152951 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152955 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152957 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152960 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152964 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152967 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152969 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152972 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152974 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152976 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152979 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152981 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152985 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152988 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.509191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.152990 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.152996 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.159627 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.159642 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.159662 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.159670 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159725 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159730 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159734 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159737 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159740 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159744 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159747 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159754 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.510067 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159757 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159760 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159766 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159769 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159772 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159775 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159777 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159781 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159783 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159786 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159789 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159791 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159794 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159796 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159799 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159801 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159804 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159806 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159809 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159812 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.513270 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159814 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159817 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159820 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159823 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159825 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159828 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159830 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159833 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159835 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159838 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159840 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159844 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159847 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159849 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159853 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159873 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159876 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159879 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159881 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159884 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.595128 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159887 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159889 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159892 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159894 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159897 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159899 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159902 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159904 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159907 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159910 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159912 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159915 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159919 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159923 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159926 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159929 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159932 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159935 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159937 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159940 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.683648 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159942 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159945 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159947 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159951 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159954 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159956 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159960 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159963 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159966 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159968 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159971 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159973 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159976 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159979 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159981 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159984 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159986 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159989 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159991 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159994 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.684318 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159997 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.159999 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160002 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160005 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160007 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160010 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160012 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160015 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160018 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160020 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160023 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.160028 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160137 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160143 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160147 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 18:37:20.684851 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160151 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160155 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160158 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160162 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160164 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160167 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160170 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160172 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160175 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160177 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160180 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160182 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160185 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160188 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160190 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160193 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160196 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160198 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160201 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160203 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 18:37:20.685328 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160206 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160208 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160211 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160214 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160216 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160219 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160222 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160224 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160227 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160229 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160232 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160234 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160237 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160241 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160243 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160247 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160250 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160252 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160255 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160257 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 18:37:20.685848 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160260 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160262 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160265 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160268 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160270 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160273 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160275 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160278 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160280 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160283 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160285 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160288 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160290 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160293 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160296 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160298 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160300 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160303 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160306 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 18:37:20.686458 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160308 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160311 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160314 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160316 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160319 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160321 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160326 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160329 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160334 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160337 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160339 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160342 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160345 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160348 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160350 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160353 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160356 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160358 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160361 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 18:37:20.686983 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160363 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160365 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160368 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160372 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160375 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160378 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160381 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160383 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160386 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160388 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160391 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160393 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160396 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160399 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160401 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160404 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160406 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 18:37:20.687532 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:20.160408 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.160413 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.160507 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.162308 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.163691 2554 server.go:1019] "Starting client certificate rotation" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.163790 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.163818 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.166192 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.172610 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.186022 2554 log.go:25] "Validated CRI v1 runtime API" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.188103 2554 log.go:25] "Validated CRI v1 image API" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.188928 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 18:37:20.688036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.191216 2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 934ea000-6e72-4461-b6a0-a5c43dc927fb:/dev/nvme0n1p4 cf6846fb-71a1-4564-86c7-686a2d0f1d23:/dev/nvme0n1p3] Jul 01 18:37:20.688416 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.191237 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.196681 2554 manager.go:216] Machine: {Timestamp:2026-07-01 18:37:20.195370154 +0000 UTC m=+0.322124571 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105304 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a57c0bf6662388425a32628419c65 SystemUUID:ec2a57c0-bf66-6238-8425-a32628419c65 BootID:a2ba27ee-093e-4d7c-82f2-0f0e4768a609 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e9:f5:e8:2b:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e9:f5:e8:2b:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:4f:43:a0:fb:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.196778 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.196850 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.197960 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.200250 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.200276 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-178.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 18:37:20.688483 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.201461 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.201474 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.201484 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.202089 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.202965 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.205932 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206050 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206075 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206091 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206100 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206110 2554 kubelet.go:398] "Adding apiserver pod source" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206118 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206466 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.206486 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.209466 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.210980 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211006 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211566 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211593 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211602 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211609 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211618 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211623 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211631 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211636 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211642 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211648 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211660 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.211665 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.212393 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.212403 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.215783 2554 server.go:1268] "Started kubelet" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.215903 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.215973 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.215982 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.220496 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.220634 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.220965 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-178.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.221038 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 18:37:20.688768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.222028 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.223562 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.223735 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-178.ec2.internal.18be3e55af0fbdbc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-178.ec2.internal,UID:ip-10-0-139-178.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-178.ec2.internal,},FirstTimestamp:2026-07-01 18:37:20.215481788 +0000 UTC m=+0.342236205,LastTimestamp:2026-07-01 18:37:20.215481788 +0000 UTC m=+0.342236205,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-178.ec2.internal,}" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.225877 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.225889 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.225943 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226343 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226423 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226452 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226550 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226560 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226650 2554 factory.go:55] Registering systemd factory Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226666 2554 factory.go:223] Registration of the systemd container factory successfully Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226843 2554 factory.go:153] Registering CRI-O factory Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226854 2554 factory.go:223] Registration of the crio container factory successfully Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226925 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226949 2554 factory.go:103] Registering Raw factory Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.226965 2554 manager.go:1196] Started watching for new ooms in manager Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.227449 2554 manager.go:349] Starting recovery of all containers Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.227919 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xlvtl" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.234816 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-178.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.237550 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xlvtl" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.239668 2554 manager.go:354] Recovery completed Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.243599 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245310 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245333 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245342 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245809 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245942 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245958 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245970 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.245974 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.246606 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:20.689827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.246627 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.246639 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.247044 2554 policy_none.go:49] "None policy: Start" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.247058 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.247070 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.248037 2554 policy_none.go:47] "Start" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.293662 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.294827 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.294883 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.294910 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.294951 2554 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.298016 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.326372 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.395627 2554 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.426786 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.442714 2554 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.527535 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.606782 2554 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616402 2554 manager.go:337] "Starting Device Plugin manager" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.616469 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616481 2554 server.go:87] "Starting device plugin registration server" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616739 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616751 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616850 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616934 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.616942 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.617502 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 18:37:20.691949 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.617539 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.717314 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.718300 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.718324 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.718336 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.718359 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-178.ec2.internal" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:20.725099 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-178.ec2.internal" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.725116 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-178.ec2.internal\": node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:20.834674 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.756822 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:22.480999 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:20.857687 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:22.481785 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.481717 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.530873 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"] Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.530944 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.532635 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.532664 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.532674 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.533952 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534060 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534084 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534628 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534653 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534663 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534628 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534721 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.534732 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.535696 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.535718 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.536398 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientMemory" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.536417 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.536431 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeHasSufficientPID" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.559313 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.559333 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91f221773be377b95c268f7c48aa904d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"91f221773be377b95c268f7c48aa904d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.559347 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.567147 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.571546 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-178.ec2.internal\" not found" node="ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.647170 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.582549 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91f221773be377b95c268f7c48aa904d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"91f221773be377b95c268f7c48aa904d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660590 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91f221773be377b95c268f7c48aa904d-config\") pod \"kube-apiserver-proxy-ip-10-0-139-178.ec2.internal\" (UID: \"91f221773be377b95c268f7c48aa904d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660623 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.660631 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83cef7a7429039a5ddcae573598994b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal\" (UID: \"83cef7a7429039a5ddcae573598994b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.682664 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-178.ec2.internal\" not found" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.684090 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.686906 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.687010 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.687047 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.687044 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.730183 2554 apiserver.go:52] "Watching apiserver" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.745858 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.746186 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-pq84p","openshift-multus/network-metrics-daemon-6hvdk","openshift-network-diagnostics/network-check-target-twz9h","openshift-network-operator/iptables-alerter-brkxf","kube-system/konnectivity-agent-hkn6f","openshift-image-registry/node-ca-5f79x","openshift-multus/multus-szl4z","openshift-ovn-kubernetes/ovnkube-node-2nnxm","kube-system/global-pull-secret-syncer-jkrhv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh","openshift-cluster-node-tuning-operator/tuned-wzxcq","openshift-dns/node-resolver-nb24j"] Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.747558 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.811077 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.748766 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.748820 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.749671 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.749699 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.749788 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.749836 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.750177 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.750229 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.750948 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.751324 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.751324 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.751637 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.751641 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hwttj\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.752193 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.754115 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.755002 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.755643 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.755780 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.755859 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kxmk9\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.756028 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.757404 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.757440 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.757513 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.758830 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.759539 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.759665 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.759788 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 18:37:22.812074 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.759948 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn5xm\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.759959 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760050 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760275 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760352 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760674 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760697 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xf8qd\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760709 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760738 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-os-release\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-626jk\" (UniqueName: \"kubernetes.io/projected/1a6078ce-b69f-40ab-b65a-c662af43cffe-kube-api-access-626jk\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760777 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760790 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpw8p\" (UniqueName: \"kubernetes.io/projected/2c9cac5a-22da-46f2-a514-049a7075ea4a-kube-api-access-gpw8p\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760813 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760813 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a7a036-b6ec-42af-bf80-b6a743d514c4-iptables-alerter-script\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760859 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-knxvr\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760816 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760883 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mw6\" (UniqueName: \"kubernetes.io/projected/a4a7a036-b6ec-42af-bf80-b6a743d514c4-kube-api-access-j7mw6\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760910 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-system-cni-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.812831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760925 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-cnibin\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760939 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760940 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8sjzs\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760974 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a7a036-b6ec-42af-bf80-b6a743d514c4-host-slash\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760971 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.760999 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.761024 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.761059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.761111 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.761647 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.762942 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lzgn6\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.762997 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.764357 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.764936 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 18:32:21 +0000 UTC" deadline="2027-12-22 10:35:34.28352118 +0000 UTC" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.764972 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12927h58m11.518551564s" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.765706 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.765732 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bj5lc\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.765751 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.766377 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.766439 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.766472 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.768517 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.768575 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.813430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.768604 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal"] Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.771159 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2k6qn\"" Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.773737 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.782597 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal"] Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.782904 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.794938 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fq467" Jul 01 18:37:22.814049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.806777 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fq467" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.850553 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861609 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861631 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-bin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861648 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-dbus\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861662 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-etc-selinux\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861709 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-systemd\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861736 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-config\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861758 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861781 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-system-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861803 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-etc-kubernetes\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861823 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861846 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-lib-modules\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnz4b\" (UniqueName: \"kubernetes.io/projected/058116fd-bb67-4bd3-9550-38ddabc7ed66-kube-api-access-wnz4b\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.925490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861894 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-os-release\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861916 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-626jk\" (UniqueName: \"kubernetes.io/projected/1a6078ce-b69f-40ab-b65a-c662af43cffe-kube-api-access-626jk\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861949 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-netd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861963 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-kubelet-config\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.861975 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-os-release\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862001 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-cnibin\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862029 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-kubelet\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862067 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-cnibin\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-kubelet\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862093 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6ea3845-6f87-4d9d-b346-d33388480adb-konnectivity-ca\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862107 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-sys\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862126 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-var-lib-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862155 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-env-overrides\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862171 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-tmp-dir\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:22.929984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862184 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-device-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862230 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-sys-fs\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862250 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-socket-dir-parent\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862267 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-hosts-file\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862289 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf08c72b-7225-4ad1-9989-82b43ce8337a-host\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862304 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862317 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-conf\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862339 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-netns\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862359 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-script-lib\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862377 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-system-cni-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862403 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862444 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-system-cni-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862448 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-daemon-config\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862484 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-systemd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.930903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862499 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-host\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-multus\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862544 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-conf-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862569 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-ovn\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-log-socket\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862614 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-bin\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862632 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862648 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3845-6f87-4d9d-b346-d33388480adb-agent-certs\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862661 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-modprobe-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862695 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862747 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862769 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-cnibin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862783 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-os-release\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862814 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-hostroot\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.941494 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.862837 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862845 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-slash\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862870 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.862907 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:23.362877659 +0000 UTC m=+1.966037556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862929 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-k8s-cni-cncf-io\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-node-log\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.862964 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovn-node-metrics-cert\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863006 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgb9v\" (UniqueName: \"kubernetes.io/projected/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-kube-api-access-qgb9v\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf08c72b-7225-4ad1-9989-82b43ce8337a-serviceca\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863057 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvzx\" (UniqueName: \"kubernetes.io/projected/cf08c72b-7225-4ad1-9989-82b43ce8337a-kube-api-access-njvzx\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863076 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-run\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863093 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a7a036-b6ec-42af-bf80-b6a743d514c4-host-slash\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863120 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.942421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863142 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysconfig\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863147 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a7a036-b6ec-42af-bf80-b6a743d514c4-host-slash\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863156 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-var-lib-kubelet\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863180 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863203 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-netns\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863229 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-multus-certs\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863263 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-etc-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863290 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-kubernetes\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863315 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-tuned\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863337 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-cni-binary-copy\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863356 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m66hr\" (UniqueName: \"kubernetes.io/projected/fb7c0baf-693d-4bc2-be0d-0346903c7574-kube-api-access-m66hr\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863377 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-systemd-units\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863397 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmk4j\" (UniqueName: \"kubernetes.io/projected/cb13e17c-1220-4060-9d8c-4d2e7bdea571-kube-api-access-nmk4j\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.943328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpw8p\" (UniqueName: \"kubernetes.io/projected/2c9cac5a-22da-46f2-a514-049a7075ea4a-kube-api-access-gpw8p\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a7a036-b6ec-42af-bf80-b6a743d514c4-iptables-alerter-script\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863504 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mw6\" (UniqueName: \"kubernetes.io/projected/a4a7a036-b6ec-42af-bf80-b6a743d514c4-kube-api-access-j7mw6\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863529 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-socket-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863631 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-tmp\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863662 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-registration-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863685 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlhm\" (UniqueName: \"kubernetes.io/projected/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kube-api-access-rwlhm\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.863705 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.864013 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a6078ce-b69f-40ab-b65a-c662af43cffe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.864830 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-binary-copy\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.864959 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.865308 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a7a036-b6ec-42af-bf80-b6a743d514c4-iptables-alerter-script\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.865563 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a6078ce-b69f-40ab-b65a-c662af43cffe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.870129 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.874145 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" Jul 01 18:37:22.944231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.878677 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.881655 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.881674 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.881686 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.881748 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:23.381733679 +0000 UTC m=+1.984893552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.881887 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-626jk\" (UniqueName: \"kubernetes.io/projected/1a6078ce-b69f-40ab-b65a-c662af43cffe-kube-api-access-626jk\") pod \"multus-additional-cni-plugins-pq84p\" (UID: \"1a6078ce-b69f-40ab-b65a-c662af43cffe\") " pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.883270 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mw6\" (UniqueName: \"kubernetes.io/projected/a4a7a036-b6ec-42af-bf80-b6a743d514c4-kube-api-access-j7mw6\") pod \"iptables-alerter-brkxf\" (UID: \"a4a7a036-b6ec-42af-bf80-b6a743d514c4\") " pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:22.945117 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.884020 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpw8p\" (UniqueName: \"kubernetes.io/projected/2c9cac5a-22da-46f2-a514-049a7075ea4a-kube-api-access-gpw8p\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.963918 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-netd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.963958 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-kubelet-config\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.963980 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-kubelet\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964013 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-kubelet\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964020 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-netd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964033 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-kubelet-config\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964039 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6ea3845-6f87-4d9d-b346-d33388480adb-konnectivity-ca\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964082 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-sys\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964081 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-kubelet\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964109 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-kubelet\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964112 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-var-lib-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964144 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-var-lib-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964148 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-sys\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964161 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-env-overrides\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.034800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964184 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-tmp-dir\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964206 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-device-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-sys-fs\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964241 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-socket-dir-parent\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964261 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-device-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964281 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-hosts-file\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964312 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf08c72b-7225-4ad1-9989-82b43ce8337a-host\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964336 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964352 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-sys-fs\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964379 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-hosts-file\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964355 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-conf\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-netns\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964446 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964430 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-socket-dir-parent\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-script-lib\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.035659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964488 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysctl-conf\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964522 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-netns\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964523 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf08c72b-7225-4ad1-9989-82b43ce8337a-host\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964568 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e6ea3845-6f87-4d9d-b346-d33388480adb-konnectivity-ca\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964590 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-tmp-dir\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964625 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-daemon-config\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964644 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-systemd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964630 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-env-overrides\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964672 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-host\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964709 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-systemd\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964716 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-multus\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964749 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-multus\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964750 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-host\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964782 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-conf-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964797 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-conf-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.036553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964811 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-ovn\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-log-socket\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-bin\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964860 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-ovn\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964892 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-log-socket\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964895 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964926 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964905 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-cni-bin\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3845-6f87-4d9d-b346-d33388480adb-agent-certs\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.964976 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-modprobe-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965024 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-cnibin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965068 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-os-release\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965105 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-hostroot\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965119 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-daemon-config\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965125 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-cnibin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.037433 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965130 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-script-lib\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965135 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-os-release\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965142 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-modprobe-d\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965128 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-slash\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965167 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-slash\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965177 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-hostroot\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965189 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965214 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-k8s-cni-cncf-io\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965230 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-host-run-ovn-kubernetes\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-node-log\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965258 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-node-log\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965259 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-k8s-cni-cncf-io\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965258 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovn-node-metrics-cert\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965639 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgb9v\" (UniqueName: \"kubernetes.io/projected/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-kube-api-access-qgb9v\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965769 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf08c72b-7225-4ad1-9989-82b43ce8337a-serviceca\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.038346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965810 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvzx\" (UniqueName: \"kubernetes.io/projected/cf08c72b-7225-4ad1-9989-82b43ce8337a-kube-api-access-njvzx\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965840 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-run\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.965910 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966365 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cf08c72b-7225-4ad1-9989-82b43ce8337a-serviceca\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966475 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-run\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966534 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-run-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966576 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysconfig\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966605 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-var-lib-kubelet\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966648 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-netns\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966694 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-multus-certs\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-etc-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-kubernetes\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966801 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-tuned\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-cni-binary-copy\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966874 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m66hr\" (UniqueName: \"kubernetes.io/projected/fb7c0baf-693d-4bc2-be0d-0346903c7574-kube-api-access-m66hr\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.039260 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-systemd-units\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966932 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmk4j\" (UniqueName: \"kubernetes.io/projected/cb13e17c-1220-4060-9d8c-4d2e7bdea571-kube-api-access-nmk4j\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.966982 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-socket-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-tmp\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-registration-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967548 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlhm\" (UniqueName: \"kubernetes.io/projected/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kube-api-access-rwlhm\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967588 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967615 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-bin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967643 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-dbus\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967672 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-etc-selinux\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967700 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-systemd\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-config\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967773 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-system-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.040146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967785 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-sysconfig\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967801 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-etc-kubernetes\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967853 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-var-lib-kubelet\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967858 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-etc-kubernetes\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.967890 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.967980 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:22.968042 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:23.468016419 +0000 UTC m=+2.071176292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968059 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e6ea3845-6f87-4d9d-b346-d33388480adb-agent-certs\") pod \"konnectivity-agent-hkn6f\" (UID: \"e6ea3845-6f87-4d9d-b346-d33388480adb\") " pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968139 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968241 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-etc-selinux\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968251 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7bae8d16-a7ec-416b-874a-385f16b4b780-dbus\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968261 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-netns\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968292 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-systemd-units\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968343 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-kubernetes\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041034 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968387 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-socket-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968395 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-run-multus-certs\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968873 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-host-var-lib-cni-bin\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968888 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4f20740d-b3f1-475a-ae47-7bb8e676afa6-registration-dir\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968952 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-system-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.968996 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-systemd\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969026 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb7c0baf-693d-4bc2-be0d-0346903c7574-multus-cni-dir\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969050 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-lib-modules\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969086 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnz4b\" (UniqueName: \"kubernetes.io/projected/058116fd-bb67-4bd3-9550-38ddabc7ed66-kube-api-access-wnz4b\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969110 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovnkube-config\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969219 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/058116fd-bb67-4bd3-9550-38ddabc7ed66-lib-modules\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969282 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb13e17c-1220-4060-9d8c-4d2e7bdea571-etc-openvswitch\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.969331 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb7c0baf-693d-4bc2-be0d-0346903c7574-cni-binary-copy\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.970626 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-etc-tuned\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.970666 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb13e17c-1220-4060-9d8c-4d2e7bdea571-ovn-node-metrics-cert\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.041942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.970886 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/058116fd-bb67-4bd3-9550-38ddabc7ed66-tmp\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.980383 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmk4j\" (UniqueName: \"kubernetes.io/projected/cb13e17c-1220-4060-9d8c-4d2e7bdea571-kube-api-access-nmk4j\") pod \"ovnkube-node-2nnxm\" (UID: \"cb13e17c-1220-4060-9d8c-4d2e7bdea571\") " pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.980589 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgb9v\" (UniqueName: \"kubernetes.io/projected/1d5ef2ef-3ddb-46b8-8f60-aaad32410974-kube-api-access-qgb9v\") pod \"node-resolver-nb24j\" (UID: \"1d5ef2ef-3ddb-46b8-8f60-aaad32410974\") " pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.980602 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvzx\" (UniqueName: \"kubernetes.io/projected/cf08c72b-7225-4ad1-9989-82b43ce8337a-kube-api-access-njvzx\") pod \"node-ca-5f79x\" (UID: \"cf08c72b-7225-4ad1-9989-82b43ce8337a\") " pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.984306 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m66hr\" (UniqueName: \"kubernetes.io/projected/fb7c0baf-693d-4bc2-be0d-0346903c7574-kube-api-access-m66hr\") pod \"multus-szl4z\" (UID: \"fb7c0baf-693d-4bc2-be0d-0346903c7574\") " pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.984331 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnz4b\" (UniqueName: \"kubernetes.io/projected/058116fd-bb67-4bd3-9550-38ddabc7ed66-kube-api-access-wnz4b\") pod \"tuned-wzxcq\" (UID: \"058116fd-bb67-4bd3-9550-38ddabc7ed66\") " pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:23.042842 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:22.984306 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlhm\" (UniqueName: \"kubernetes.io/projected/4f20740d-b3f1-475a-ae47-7bb8e676afa6-kube-api-access-rwlhm\") pod \"aws-ebs-csi-driver-node-phxsh\" (UID: \"4f20740d-b3f1-475a-ae47-7bb8e676afa6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.145214 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.100700 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pq84p" Jul 01 18:37:23.145214 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.108427 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:23.145214 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.142258 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-brkxf" Jul 01 18:37:23.251821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.164171 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:23.251821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.180693 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5f79x" Jul 01 18:37:23.251821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.192959 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szl4z" Jul 01 18:37:23.251821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.217402 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:23.251821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.222871 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" Jul 01 18:37:23.467232 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.372722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:23.467232 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.372822 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:23.467232 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.372869 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:24.372856563 +0000 UTC m=+2.976016435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.473804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.473877 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.473943 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.473985 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.473999 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:24.47398454 +0000 UTC m=+3.077144412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.474000 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.474015 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:23.475332 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:23.474038 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:24.474031751 +0000 UTC m=+3.077191623 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:23.811847 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.811127 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:32:22 +0000 UTC" deadline="2028-04-04 17:35:17.392795502 +0000 UTC" Jul 01 18:37:23.811847 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.811151 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15430h57m53.581647186s" Jul 01 18:37:24.006226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.926346 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:24.006226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:23.944274 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:24.118587 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.118564 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" Jul 01 18:37:24.123907 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.123890 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nb24j" Jul 01 18:37:24.310300 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.310270 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a7a036_b6ec_42af_bf80_b6a743d514c4.slice/crio-127ac208130286e95f56c3bd37c21a66fcc6bf30fc2724cde97109bcc3b77bb3 WatchSource:0}: Error finding container 127ac208130286e95f56c3bd37c21a66fcc6bf30fc2724cde97109bcc3b77bb3: Status 404 returned error can't find the container with id 127ac208130286e95f56c3bd37c21a66fcc6bf30fc2724cde97109bcc3b77bb3 Jul 01 18:37:24.311063 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.311036 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83cef7a7429039a5ddcae573598994b6.slice/crio-e45d16b614b17ca557bc2115f847406c86c38fb6f17f950f0a91bc526ee36c21 WatchSource:0}: Error finding container e45d16b614b17ca557bc2115f847406c86c38fb6f17f950f0a91bc526ee36c21: Status 404 returned error can't find the container with id e45d16b614b17ca557bc2115f847406c86c38fb6f17f950f0a91bc526ee36c21 Jul 01 18:37:24.311752 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.311729 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f221773be377b95c268f7c48aa904d.slice/crio-9729509ebecc785fbb91d2f9d832c30f3f4db1c66e9d665df9ad7d0d8ade5278 WatchSource:0}: Error finding container 9729509ebecc785fbb91d2f9d832c30f3f4db1c66e9d665df9ad7d0d8ade5278: Status 404 returned error can't find the container with id 9729509ebecc785fbb91d2f9d832c30f3f4db1c66e9d665df9ad7d0d8ade5278 Jul 01 18:37:24.312512 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.312442 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6078ce_b69f_40ab_b65a_c662af43cffe.slice/crio-85d39b3323dec6ce8223715fa0bffecbe3eb4aa66619e6ad961b44ef8af47079 WatchSource:0}: Error finding container 85d39b3323dec6ce8223715fa0bffecbe3eb4aa66619e6ad961b44ef8af47079: Status 404 returned error can't find the container with id 85d39b3323dec6ce8223715fa0bffecbe3eb4aa66619e6ad961b44ef8af47079 Jul 01 18:37:24.315004 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.314988 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:37:24.380670 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.380642 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:24.380804 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.380767 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:24.380868 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.380836 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:26.380817542 +0000 UTC m=+4.983977433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:24.447012 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.446984 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7c0baf_693d_4bc2_be0d_0346903c7574.slice/crio-5eb8588eb336d7e9d38f1759bf882233ff3c618aa4743c79b83c0a4078c1847b WatchSource:0}: Error finding container 5eb8588eb336d7e9d38f1759bf882233ff3c618aa4743c79b83c0a4078c1847b: Status 404 returned error can't find the container with id 5eb8588eb336d7e9d38f1759bf882233ff3c618aa4743c79b83c0a4078c1847b Jul 01 18:37:24.454030 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.454005 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ea3845_6f87_4d9d_b346_d33388480adb.slice/crio-2ba06950b245936bd95b820e93ce0fdb3fefd7fad9df12f4acfa83a1b804f30a WatchSource:0}: Error finding container 2ba06950b245936bd95b820e93ce0fdb3fefd7fad9df12f4acfa83a1b804f30a: Status 404 returned error can't find the container with id 2ba06950b245936bd95b820e93ce0fdb3fefd7fad9df12f4acfa83a1b804f30a Jul 01 18:37:24.469427 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.469405 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5ef2ef_3ddb_46b8_8f60_aaad32410974.slice/crio-cbedf6bf32fc11303a319203e68f16a00a7b335a4e93c1e741dd98ade501408c WatchSource:0}: Error finding container cbedf6bf32fc11303a319203e68f16a00a7b335a4e93c1e741dd98ade501408c: Status 404 returned error can't find the container with id cbedf6bf32fc11303a319203e68f16a00a7b335a4e93c1e741dd98ade501408c Jul 01 18:37:24.481893 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.481871 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:24.481994 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.481911 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482338 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482375 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482389 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482438 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482471 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:26.482432794 +0000 UTC m=+5.085592681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:24.484947 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.482531 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:26.482502944 +0000 UTC m=+5.085662826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:24.488276 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.488243 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f20740d_b3f1_475a_ae47_7bb8e676afa6.slice/crio-035eac201c96e63012d7ddb88e945cb5d871effce1225e5d0463a98268ca2b49 WatchSource:0}: Error finding container 035eac201c96e63012d7ddb88e945cb5d871effce1225e5d0463a98268ca2b49: Status 404 returned error can't find the container with id 035eac201c96e63012d7ddb88e945cb5d871effce1225e5d0463a98268ca2b49 Jul 01 18:37:24.492713 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.492688 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb13e17c_1220_4060_9d8c_4d2e7bdea571.slice/crio-63d9157cfcda501ce188705b7ab6138f83ad5ea8a0ed7b0700eb87c2c455cc40 WatchSource:0}: Error finding container 63d9157cfcda501ce188705b7ab6138f83ad5ea8a0ed7b0700eb87c2c455cc40: Status 404 returned error can't find the container with id 63d9157cfcda501ce188705b7ab6138f83ad5ea8a0ed7b0700eb87c2c455cc40 Jul 01 18:37:24.495026 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:24.495007 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058116fd_bb67_4bd3_9550_38ddabc7ed66.slice/crio-21c612d8a7a1967fe1f82f64a96148df902d9f72befe47ec30295a671d7748e0 WatchSource:0}: Error finding container 21c612d8a7a1967fe1f82f64a96148df902d9f72befe47ec30295a671d7748e0: Status 404 returned error can't find the container with id 21c612d8a7a1967fe1f82f64a96148df902d9f72befe47ec30295a671d7748e0 Jul 01 18:37:24.812127 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.812020 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 18:32:22 +0000 UTC" deadline="2027-12-31 11:39:53.177774009 +0000 UTC" Jul 01 18:37:24.812127 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.812084 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13145h2m28.36569472s" Jul 01 18:37:24.819657 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.819634 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:24.819792 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.819753 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:24.820189 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.820146 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:24.820282 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.820250 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:24.820337 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.820308 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:24.820389 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:24.820373 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:24.828532 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.828482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nb24j" event={"ID":"1d5ef2ef-3ddb-46b8-8f60-aaad32410974","Type":"ContainerStarted","Data":"cbedf6bf32fc11303a319203e68f16a00a7b335a4e93c1e741dd98ade501408c"} Jul 01 18:37:24.836254 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.836227 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szl4z" event={"ID":"fb7c0baf-693d-4bc2-be0d-0346903c7574","Type":"ContainerStarted","Data":"5eb8588eb336d7e9d38f1759bf882233ff3c618aa4743c79b83c0a4078c1847b"} Jul 01 18:37:24.838682 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.838654 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" event={"ID":"91f221773be377b95c268f7c48aa904d","Type":"ContainerStarted","Data":"9729509ebecc785fbb91d2f9d832c30f3f4db1c66e9d665df9ad7d0d8ade5278"} Jul 01 18:37:24.840593 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.840568 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"83cef7a7429039a5ddcae573598994b6","Type":"ContainerStarted","Data":"e45d16b614b17ca557bc2115f847406c86c38fb6f17f950f0a91bc526ee36c21"} Jul 01 18:37:24.845083 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.845058 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5f79x" event={"ID":"cf08c72b-7225-4ad1-9989-82b43ce8337a","Type":"ContainerStarted","Data":"6323c38080117e03cc8bc381b2c99a991cb7563e5a27c444a4bf5e54f916eaad"} Jul 01 18:37:24.847473 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.847436 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" event={"ID":"4f20740d-b3f1-475a-ae47-7bb8e676afa6","Type":"ContainerStarted","Data":"035eac201c96e63012d7ddb88e945cb5d871effce1225e5d0463a98268ca2b49"} Jul 01 18:37:24.851185 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.851120 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkn6f" event={"ID":"e6ea3845-6f87-4d9d-b346-d33388480adb","Type":"ContainerStarted","Data":"2ba06950b245936bd95b820e93ce0fdb3fefd7fad9df12f4acfa83a1b804f30a"} Jul 01 18:37:24.859070 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.859045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerStarted","Data":"85d39b3323dec6ce8223715fa0bffecbe3eb4aa66619e6ad961b44ef8af47079"} Jul 01 18:37:24.863130 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.863107 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-brkxf" event={"ID":"a4a7a036-b6ec-42af-bf80-b6a743d514c4","Type":"ContainerStarted","Data":"127ac208130286e95f56c3bd37c21a66fcc6bf30fc2724cde97109bcc3b77bb3"} Jul 01 18:37:24.872357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.872333 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" event={"ID":"058116fd-bb67-4bd3-9550-38ddabc7ed66","Type":"ContainerStarted","Data":"21c612d8a7a1967fe1f82f64a96148df902d9f72befe47ec30295a671d7748e0"} Jul 01 18:37:24.878478 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:24.878435 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"63d9157cfcda501ce188705b7ab6138f83ad5ea8a0ed7b0700eb87c2c455cc40"} Jul 01 18:37:25.659683 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:25.659571 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 18:37:26.400054 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.399956 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:26.400516 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.400118 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:26.400516 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.400181 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:30.400161364 +0000 UTC m=+9.003321242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.500582 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.500674 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.500819 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.500836 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.500849 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.500905 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:30.500886085 +0000 UTC m=+9.104045971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.501284 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:26.501368 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.501334 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:30.501319771 +0000 UTC m=+9.104479647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:26.818961 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.818885 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:26.819121 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.819017 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:26.819396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.819377 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:26.819528 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.819504 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:26.819607 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:26.819561 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:26.819658 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:26.819626 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:28.819646 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:28.819613 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:28.819995 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:28.819756 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:28.819995 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:28.819875 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:28.819995 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:28.819994 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:28.820156 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:28.820041 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:28.820156 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:28.820110 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:30.434858 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.434523 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:30.434858 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.434758 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:30.434858 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.434826 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:38.434805947 +0000 UTC m=+17.037965839 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.534899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.534985 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535007 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535068 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:38.535051289 +0000 UTC m=+17.138211179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535088 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535101 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535113 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:30.535185 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.535150 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:38.535139107 +0000 UTC m=+17.138298997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.819509 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.819635 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.820015 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.820110 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:30.820171 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:30.820330 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:30.820249 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:32.818819 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:32.818786 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:32.819261 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:32.818827 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:32.819261 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:32.818825 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:32.819261 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:32.818915 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:32.819261 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:32.819037 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:32.819261 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:32.819115 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:34.819729 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:34.819692 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:34.820180 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:34.819692 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:34.820180 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:34.819816 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:34.820180 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:34.819710 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:34.820180 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:34.819863 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:34.820180 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:34.819955 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:36.818730 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:36.818690 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:36.819183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:36.818689 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:36.819183 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:36.818828 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:36.819183 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:36.818920 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:36.819183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:36.818690 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:36.819183 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:36.819020 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:38.496352 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.496316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:38.496843 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.496743 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:38.496904 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.496846 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:37:54.496813693 +0000 UTC m=+33.099973579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:38.597300 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.597260 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:38.597500 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.597359 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:38.597500 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597374 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:38.597500 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597440 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:54.597419083 +0000 UTC m=+33.200578972 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:38.597636 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597522 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:38.597636 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597539 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:38.597636 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597552 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:38.597636 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.597603 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:37:54.597589043 +0000 UTC m=+33.200748927 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:38.819341 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.819256 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:38.819341 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.819298 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:38.819341 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:38.819298 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:38.819630 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.819377 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:38.819630 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.819517 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:38.819630 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:38.819591 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:40.819771 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:40.819573 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:40.820238 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:40.819573 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:40.820238 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:40.819876 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:40.820238 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:40.819599 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:40.820238 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:40.819933 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:40.820238 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:40.820012 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:42.819206 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:42.819168 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:42.819206 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:42.819207 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:42.819715 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:42.819284 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:42.819715 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:42.819304 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:42.819715 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:42.819368 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:42.819715 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:42.819435 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:44.819200 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.819165 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:44.819751 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.819209 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:44.819751 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.819183 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:44.819751 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:44.819307 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:44.819751 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:44.819372 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:44.819751 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:44.819440 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:44.937382 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.937348 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" event={"ID":"058116fd-bb67-4bd3-9550-38ddabc7ed66","Type":"ContainerStarted","Data":"21f1e9decf6e660eecf0467520392d3ca01f1bc581e54f3f2c1081f743a6ee1c"} Jul 01 18:37:44.940367 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.940315 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"dd7f1a41c4837a8306aff0fb118eac02e9db3294ed15fa9ddac0e94aefd91c9c"} Jul 01 18:37:44.942022 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.942003 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szl4z" event={"ID":"fb7c0baf-693d-4bc2-be0d-0346903c7574","Type":"ContainerStarted","Data":"b26921429d4baca13264f5cda5450da63bdd08354d2745308a85dc04ee17ad0c"} Jul 01 18:37:44.943963 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.943573 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" event={"ID":"91f221773be377b95c268f7c48aa904d","Type":"ContainerStarted","Data":"8c3d6e562e15fd26baf135233a00cfdde212d4cad63cbb28c42383657953c7b5"} Jul 01 18:37:44.960447 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.960399 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wzxcq" podStartSLOduration=3.167495844 podStartE2EDuration="22.960379383s" totalImagesPullingTime="19.792883539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:44.95996718 +0000 UTC m=+23.563127076" watchObservedRunningTime="2026-07-01 18:37:44.960379383 +0000 UTC m=+23.563539276" Jul 01 18:37:44.989536 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:44.989429 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-szl4z" podStartSLOduration=3.139860212 podStartE2EDuration="22.989408999s" totalImagesPullingTime="19.849548787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:44.988850164 +0000 UTC m=+23.592010059" watchObservedRunningTime="2026-07-01 18:37:44.989408999 +0000 UTC m=+23.592568895" Jul 01 18:37:45.948423 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948133 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948752 2554 generic.go:356] "Generic (PLEG): container finished" podID="cb13e17c-1220-4060-9d8c-4d2e7bdea571" containerID="6eaf9ba865cdd3fd27c094cfa35c22f69800a4014df36fe2b907d5909b9953b1" exitCode=1 Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948829 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"323df178328b8f8e546b8a80044b894ba1c85005807f971a0114b7298ec4d845"} Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948894 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"b7f5a54d031ba71b366f82fdcfc8e1849dae37d5e52e8f762c2f453aaf510817"} Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948929 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"c62e82db5c5c575536d14e7617a47d4d2dcfe0a842399222e1f827a91a6fb0a7"} Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948957 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"b1bfd7a7437171efaa697e59493c606d3cc5ff68713a8274c260a75c95bcab8d"} Jul 01 18:37:45.949139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.948997 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerDied","Data":"6eaf9ba865cdd3fd27c094cfa35c22f69800a4014df36fe2b907d5909b9953b1"} Jul 01 18:37:45.950226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.950206 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nb24j" event={"ID":"1d5ef2ef-3ddb-46b8-8f60-aaad32410974","Type":"ContainerStarted","Data":"9ca3ba5a6989a4ae6d86aadc3cb99eff28e08187bb074c870e667eb04e258aab"} Jul 01 18:37:45.951630 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.951609 2554 generic.go:356] "Generic (PLEG): container finished" podID="83cef7a7429039a5ddcae573598994b6" containerID="ffc2ade05c5d6bff4bf5b121bc36930628660a90ad07fc3aa901b06a33c41425" exitCode=0 Jul 01 18:37:45.951716 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.951676 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"83cef7a7429039a5ddcae573598994b6","Type":"ContainerDied","Data":"ffc2ade05c5d6bff4bf5b121bc36930628660a90ad07fc3aa901b06a33c41425"} Jul 01 18:37:45.952988 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.952966 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5f79x" event={"ID":"cf08c72b-7225-4ad1-9989-82b43ce8337a","Type":"ContainerStarted","Data":"7d8b6939d02008fdb90a3e5cae547e2f17e1d6b625fb20793f7f06e52beb87cd"} Jul 01 18:37:45.954198 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.954181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" event={"ID":"4f20740d-b3f1-475a-ae47-7bb8e676afa6","Type":"ContainerStarted","Data":"dc8304922682e77d07b72124da8ba831a5b4127fc56f837350f1bb2422f87f23"} Jul 01 18:37:45.955395 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.955378 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkn6f" event={"ID":"e6ea3845-6f87-4d9d-b346-d33388480adb","Type":"ContainerStarted","Data":"0239da662537c2eff4d63815a0eda6a2d714ecd2e0c559e2bcbde9388f465fb3"} Jul 01 18:37:45.956783 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.956763 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="bb613cabbea5dc7b8821cba7701052e0db2579011db7974dad5a87e0a9004097" exitCode=0 Jul 01 18:37:45.956872 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.956825 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"bb613cabbea5dc7b8821cba7701052e0db2579011db7974dad5a87e0a9004097"} Jul 01 18:37:45.959909 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.959887 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-brkxf" event={"ID":"a4a7a036-b6ec-42af-bf80-b6a743d514c4","Type":"ContainerStarted","Data":"836f21255a05fc09c0086cb23079067d7e331753dcf8e589fe2ccdd8b4bb4f49"} Jul 01 18:37:45.975511 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.975473 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-178.ec2.internal" podStartSLOduration=23.975445446 podStartE2EDuration="23.975445446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:45.011907185 +0000 UTC m=+23.615067078" watchObservedRunningTime="2026-07-01 18:37:45.975445446 +0000 UTC m=+24.578605337" Jul 01 18:37:45.996524 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.996488 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nb24j" podStartSLOduration=4.186797836 podStartE2EDuration="23.996475243s" totalImagesPullingTime="19.809677407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:45.975687883 +0000 UTC m=+24.578847776" watchObservedRunningTime="2026-07-01 18:37:45.996475243 +0000 UTC m=+24.599635141" Jul 01 18:37:45.996684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:45.996661 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-brkxf" podStartSLOduration=4.02416578 podStartE2EDuration="23.996655334s" totalImagesPullingTime="19.972489554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:45.996269729 +0000 UTC m=+24.599429624" watchObservedRunningTime="2026-07-01 18:37:45.996655334 +0000 UTC m=+24.599815228" Jul 01 18:37:46.028371 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.028329 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5f79x" podStartSLOduration=4.3522039 podStartE2EDuration="24.028313658s" totalImagesPullingTime="19.676109758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:46.02780592 +0000 UTC m=+24.630965817" watchObservedRunningTime="2026-07-01 18:37:46.028313658 +0000 UTC m=+24.631473551" Jul 01 18:37:46.045894 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.045852 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hkn6f" podStartSLOduration=4.219609343 podStartE2EDuration="24.045840762s" totalImagesPullingTime="19.826231419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:46.045708619 +0000 UTC m=+24.648868513" watchObservedRunningTime="2026-07-01 18:37:46.045840762 +0000 UTC m=+24.649000657" Jul 01 18:37:46.172157 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.172128 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkn6f_e6ea3845-6f87-4d9d-b346-d33388480adb/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:46.262286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.262260 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkn6f_e6ea3845-6f87-4d9d-b346-d33388480adb/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:46.333905 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.333875 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkn6f_e6ea3845-6f87-4d9d-b346-d33388480adb/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:46.450981 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.450961 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 18:37:46.819755 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.819716 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:46.819971 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.819762 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:46.819971 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.819827 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:46.819971 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:46.819844 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:46.819971 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:46.819897 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:46.820124 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:46.819986 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:46.963954 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.963918 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" event={"ID":"83cef7a7429039a5ddcae573598994b6","Type":"ContainerStarted","Data":"bebf6f6ff07bef506675bb3351d533981d36ba68ba628335ecd2d637bee98527"} Jul 01 18:37:46.965947 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.965920 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" event={"ID":"4f20740d-b3f1-475a-ae47-7bb8e676afa6","Type":"ContainerStarted","Data":"be72dba3ff780e65d4afef7bcfb60a70b2b029af2d69c48592ce8fa4145696c3"} Jul 01 18:37:46.984303 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:46.984244 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-178.ec2.internal" podStartSLOduration=24.984229818 podStartE2EDuration="24.984229818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:46.983621671 +0000 UTC m=+25.586781566" watchObservedRunningTime="2026-07-01 18:37:46.984229818 +0000 UTC m=+25.587389712" Jul 01 18:37:47.151656 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.151535 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T18:37:46.450979933Z","UUID":"dd6fe597-1fda-441e-b327-a2d27c9b2034","Handler":null,"Name":"","Endpoint":""} Jul 01 18:37:47.153168 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.153130 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 18:37:47.153286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.153175 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 18:37:47.392409 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.392373 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:47.393007 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.392989 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:47.971583 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.971557 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:37:47.972026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.971984 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"6a2b1a05addb86e7ee5a9321f17426412e3ca6aebddda44af6d9ab5c11fb43f6"} Jul 01 18:37:47.972521 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.972496 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:47.973068 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:47.973046 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hkn6f" Jul 01 18:37:48.063883 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:48.063854 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hkn6f_e6ea3845-6f87-4d9d-b346-d33388480adb/konnectivity-agent/0.log" limit=1024 Jul 01 18:37:48.819629 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:48.819382 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:48.819821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:48.819383 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:48.819821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:48.819383 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:48.819821 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:48.819743 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:48.819953 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:48.819822 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:48.819953 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:48.819887 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:48.976847 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:48.976806 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" event={"ID":"4f20740d-b3f1-475a-ae47-7bb8e676afa6","Type":"ContainerStarted","Data":"835ff20d0bfdb7736772d08e5c4cd6d083e3d89b3f347e7de3d6ce997ad7aa60"} Jul 01 18:37:49.001527 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:49.001480 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phxsh" podStartSLOduration=4.406106822 podStartE2EDuration="27.001445915s" totalImagesPullingTime="22.595339093s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:49.001261651 +0000 UTC m=+27.604421572" watchObservedRunningTime="2026-07-01 18:37:49.001445915 +0000 UTC m=+27.604605815" Jul 01 18:37:50.818924 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.818738 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:50.819651 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.818783 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:50.819651 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:50.819009 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:50.819651 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.818783 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:50.819651 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:50.819084 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:50.819651 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:50.819198 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:50.984089 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984061 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:37:50.984430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984409 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"e074cf84a5da01c29bb197b2c4d6b40920e919cd58ccdc30b4ea9177d4757731"} Jul 01 18:37:50.984589 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984567 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:50.984709 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984637 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:50.984709 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984688 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:50.984825 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.984812 2554 scope.go:117] "RemoveContainer" containerID="6eaf9ba865cdd3fd27c094cfa35c22f69800a4014df36fe2b907d5909b9953b1" Jul 01 18:37:50.986808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.986776 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="f34e5ae0f4ed38c5f82d933f885956a05c9d4183c1a16821f7dde00f33306be6" exitCode=0 Jul 01 18:37:50.986916 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:50.986812 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"f34e5ae0f4ed38c5f82d933f885956a05c9d4183c1a16821f7dde00f33306be6"} Jul 01 18:37:51.001352 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:51.001333 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:51.001484 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:51.001471 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:37:51.993241 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:51.993215 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:37:51.993711 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:51.993658 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" event={"ID":"cb13e17c-1220-4060-9d8c-4d2e7bdea571","Type":"ContainerStarted","Data":"0a16c57d1082769e4f9307d7a1809aea0d3bfe50d5eea7a179558e1134b056fd"} Jul 01 18:37:52.029859 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.029245 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" podStartSLOduration=9.700614063 podStartE2EDuration="30.029222801s" totalImagesPullingTime="20.328608738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:37:52.028305567 +0000 UTC m=+30.631465460" watchObservedRunningTime="2026-07-01 18:37:52.029222801 +0000 UTC m=+30.632382698" Jul 01 18:37:52.180166 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:52.180140 2554 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6078ce_b69f_40ab_b65a_c662af43cffe.slice/crio-2a0b2a03e17e71ff661a065646f08f6e1b54833d0a2ac5b3a82ebc58b59a2406.scope\": RecentStats: unable to find data in memory cache]" Jul 01 18:37:52.613021 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.612992 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-twz9h"] Jul 01 18:37:52.613185 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.613119 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:52.613232 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:52.613197 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:52.621630 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.621603 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkrhv"] Jul 01 18:37:52.621766 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.621717 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:52.621836 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:52.621816 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:52.622177 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.622152 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6hvdk"] Jul 01 18:37:52.622273 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.622260 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:52.622369 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:52.622351 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:52.997879 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.997847 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="2a0b2a03e17e71ff661a065646f08f6e1b54833d0a2ac5b3a82ebc58b59a2406" exitCode=0 Jul 01 18:37:52.998253 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:52.997936 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"2a0b2a03e17e71ff661a065646f08f6e1b54833d0a2ac5b3a82ebc58b59a2406"} Jul 01 18:37:54.514922 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.514675 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:54.515326 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.514811 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:54.515326 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.515060 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:38:26.515042464 +0000 UTC m=+65.118202339 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 18:37:54.615900 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.615868 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:54.616028 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.615935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:54.616108 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616087 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:54.616226 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616159 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret podName:7bae8d16-a7ec-416b-874a-385f16b4b780 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:26.616143921 +0000 UTC m=+65.219303820 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret") pod "global-pull-secret-syncer-jkrhv" (UID: "7bae8d16-a7ec-416b-874a-385f16b4b780") : object "kube-system"/"original-pull-secret" not registered Jul 01 18:37:54.616226 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616086 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 18:37:54.616351 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616207 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 18:37:54.616351 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616253 2554 projected.go:196] Error preparing data for projected volume kube-api-access-ndlpn for pod openshift-network-diagnostics/network-check-target-twz9h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:54.616351 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.616318 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn podName:1bdf0907-9f42-406f-a37e-49e5cc80e144 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:26.616299268 +0000 UTC m=+65.219459155 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ndlpn" (UniqueName: "kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn") pod "network-check-target-twz9h" (UID: "1bdf0907-9f42-406f-a37e-49e5cc80e144") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 18:37:54.818787 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.818703 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:54.818932 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.818703 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:54.818932 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.818835 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:54.818932 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:54.818703 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:54.818932 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.818889 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:54.819133 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:54.818949 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:55.004911 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:55.004881 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="bc121158779e8de5300da0aa7dc9d11189174eca87bc84a40a14d03e7c09ecb0" exitCode=0 Jul 01 18:37:55.005078 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:55.004929 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"bc121158779e8de5300da0aa7dc9d11189174eca87bc84a40a14d03e7c09ecb0"} Jul 01 18:37:56.819588 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:56.819551 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:56.819588 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:56.819584 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:56.820199 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:56.819584 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:56.820199 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:56.819679 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrhv" podUID="7bae8d16-a7ec-416b-874a-385f16b4b780" Jul 01 18:37:56.820199 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:56.819781 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6hvdk" podUID="2c9cac5a-22da-46f2-a514-049a7075ea4a" Jul 01 18:37:56.820199 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:56.819850 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-twz9h" podUID="1bdf0907-9f42-406f-a37e-49e5cc80e144" Jul 01 18:37:58.706023 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.705995 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-178.ec2.internal" event="NodeReady" Jul 01 18:37:58.706635 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.706118 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 18:37:58.761300 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.761268 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zr9jd"] Jul 01 18:37:58.763908 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.763875 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.766236 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.766207 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7sn7m\"" Jul 01 18:37:58.766380 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.766362 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b72ng"] Jul 01 18:37:58.766604 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.766576 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 18:37:58.766604 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.766596 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 18:37:58.768418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.768366 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:58.770473 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.770432 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ttz69\"" Jul 01 18:37:58.770564 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.770538 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 18:37:58.770740 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.770727 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 18:37:58.770831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.770774 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 18:37:58.778086 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.778067 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zr9jd"] Jul 01 18:37:58.781329 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.781264 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b72ng"] Jul 01 18:37:58.819350 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.819317 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:37:58.819515 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.819427 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:37:58.819515 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.819437 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:37:58.822374 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822351 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 18:37:58.822374 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822369 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 18:37:58.822571 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822408 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 18:37:58.822571 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822475 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lwv7s\"" Jul 01 18:37:58.822722 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822703 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vz9\"" Jul 01 18:37:58.822886 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.822864 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 18:37:58.847957 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.847936 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fce2c7b4-6610-44c0-882d-d816c734aa6e-config-volume\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.848055 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.847965 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szf2k\" (UniqueName: \"kubernetes.io/projected/fce2c7b4-6610-44c0-882d-d816c734aa6e-kube-api-access-szf2k\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.848055 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.847999 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.848157 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.848059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce2c7b4-6610-44c0-882d-d816c734aa6e-tmp-dir\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.949308 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949276 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e455c3a6-5452-4971-8b72-072592a96d92-cert\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:58.949491 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949361 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fce2c7b4-6610-44c0-882d-d816c734aa6e-config-volume\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.949491 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949390 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szf2k\" (UniqueName: \"kubernetes.io/projected/fce2c7b4-6610-44c0-882d-d816c734aa6e-kube-api-access-szf2k\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.949491 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949428 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.949491 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949485 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce2c7b4-6610-44c0-882d-d816c734aa6e-tmp-dir\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.949701 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949508 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2xnp\" (UniqueName: \"kubernetes.io/projected/e455c3a6-5452-4971-8b72-072592a96d92-kube-api-access-x2xnp\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:58.949701 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:58.949588 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:58.949701 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:58.949671 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:37:59.449648383 +0000 UTC m=+38.052808276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:37:58.949857 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949825 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fce2c7b4-6610-44c0-882d-d816c734aa6e-tmp-dir\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.950023 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.949999 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fce2c7b4-6610-44c0-882d-d816c734aa6e-config-volume\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:58.965482 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:58.965405 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szf2k\" (UniqueName: \"kubernetes.io/projected/fce2c7b4-6610-44c0-882d-d816c734aa6e-kube-api-access-szf2k\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:59.050659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.050626 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2xnp\" (UniqueName: \"kubernetes.io/projected/e455c3a6-5452-4971-8b72-072592a96d92-kube-api-access-x2xnp\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:59.050838 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.050681 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e455c3a6-5452-4971-8b72-072592a96d92-cert\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:59.053071 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.053048 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e455c3a6-5452-4971-8b72-072592a96d92-cert\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:59.063749 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.063728 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2xnp\" (UniqueName: \"kubernetes.io/projected/e455c3a6-5452-4971-8b72-072592a96d92-kube-api-access-x2xnp\") pod \"ingress-canary-b72ng\" (UID: \"e455c3a6-5452-4971-8b72-072592a96d92\") " pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:59.083691 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.083663 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b72ng" Jul 01 18:37:59.453130 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.453092 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:37:59.453299 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:59.453236 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:37:59.453360 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:37:59.453303 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:38:00.453283273 +0000 UTC m=+39.056443158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:37:59.921207 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:37:59.921038 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b72ng"] Jul 01 18:37:59.925414 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:37:59.925391 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode455c3a6_5452_4971_8b72_072592a96d92.slice/crio-ee1d3c672088260a93ca5005a8c8bcf2845043cd492fbad9129b0361baa7a82d WatchSource:0}: Error finding container ee1d3c672088260a93ca5005a8c8bcf2845043cd492fbad9129b0361baa7a82d: Status 404 returned error can't find the container with id ee1d3c672088260a93ca5005a8c8bcf2845043cd492fbad9129b0361baa7a82d Jul 01 18:38:00.020286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:00.020248 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerStarted","Data":"a57138f3b3dc7264f50ed3c69e1d398e79d30f779486eee392b50a907e289ac2"} Jul 01 18:38:00.022169 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:00.022148 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b72ng" event={"ID":"e455c3a6-5452-4971-8b72-072592a96d92","Type":"ContainerStarted","Data":"ee1d3c672088260a93ca5005a8c8bcf2845043cd492fbad9129b0361baa7a82d"} Jul 01 18:38:00.460244 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:00.460166 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:38:00.460400 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:00.460376 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:00.460501 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:00.460488 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:38:02.460442109 +0000 UTC m=+41.063601992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:38:01.029070 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:01.029031 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="a57138f3b3dc7264f50ed3c69e1d398e79d30f779486eee392b50a907e289ac2" exitCode=0 Jul 01 18:38:01.029612 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:01.029108 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"a57138f3b3dc7264f50ed3c69e1d398e79d30f779486eee392b50a907e289ac2"} Jul 01 18:38:02.035537 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:02.035505 2554 generic.go:356] "Generic (PLEG): container finished" podID="1a6078ce-b69f-40ab-b65a-c662af43cffe" containerID="825082b3b128730ccc66553cdcdc8297576042acae128b9aea8e6b772d61591c" exitCode=0 Jul 01 18:38:02.036001 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:02.035571 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerDied","Data":"825082b3b128730ccc66553cdcdc8297576042acae128b9aea8e6b772d61591c"} Jul 01 18:38:02.476804 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:02.476602 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:38:02.476992 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:02.476749 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:02.476992 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:02.476897 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:38:06.476875361 +0000 UTC m=+45.080035253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:38:03.041119 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:03.041084 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pq84p" event={"ID":"1a6078ce-b69f-40ab-b65a-c662af43cffe","Type":"ContainerStarted","Data":"82006e5bd3c150f60003962494d47574cf132bba44aa3ca7466856cdcb6757ca"} Jul 01 18:38:03.042506 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:03.042483 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b72ng" event={"ID":"e455c3a6-5452-4971-8b72-072592a96d92","Type":"ContainerStarted","Data":"170fcae3c6c886096670601b62be599f784284cf41311b665f11365f10412b15"} Jul 01 18:38:03.081884 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:03.081839 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pq84p" podStartSLOduration=9.950344875999999 podStartE2EDuration="41.081827371s" totalImagesPullingTime="31.131482495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:03.080789933 +0000 UTC m=+41.683949827" watchObservedRunningTime="2026-07-01 18:38:03.081827371 +0000 UTC m=+41.684987266" Jul 01 18:38:03.111579 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:03.111532 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b72ng" podStartSLOduration=2.732673417 podStartE2EDuration="5.111517592s" totalImagesPullingTime="2.378844175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:03.110856006 +0000 UTC m=+41.714015901" watchObservedRunningTime="2026-07-01 18:38:03.111517592 +0000 UTC m=+41.714677487" Jul 01 18:38:06.504443 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:06.504408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:38:06.504831 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:06.504527 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:06.504831 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:06.504577 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:38:14.504564624 +0000 UTC m=+53.107724497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:38:14.557310 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:14.557273 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:38:14.557777 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:14.557444 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:14.557777 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:14.557544 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:38:30.55752601 +0000 UTC m=+69.160685882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:38:23.008696 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:23.008667 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2nnxm" Jul 01 18:38:24.293127 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.293088 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z"] Jul 01 18:38:24.327864 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.327841 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z"] Jul 01 18:38:24.328018 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.327930 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.330719 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.330697 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 01 18:38:24.330805 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.330726 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.331725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.331700 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qncbk\"" Jul 01 18:38:24.331835 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.331710 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.331835 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.331807 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 01 18:38:24.401396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.401338 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr"] Jul 01 18:38:24.422307 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.422279 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn"] Jul 01 18:38:24.422426 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.422403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.425448 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.425425 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9fscb\"" Jul 01 18:38:24.425766 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.425748 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.425868 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.425777 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 01 18:38:24.425959 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.425942 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.426032 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.426015 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 01 18:38:24.426727 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.426705 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.426832 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.426789 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82wgz\" (UniqueName: \"kubernetes.io/projected/ae045f82-c67a-48d1-9731-f3f999363168-kube-api-access-82wgz\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.426900 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.426849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae045f82-c67a-48d1-9731-f3f999363168-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.446200 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.446184 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-45t9c"] Jul 01 18:38:24.446324 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.446311 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.448730 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.448716 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 01 18:38:24.448834 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.448732 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 01 18:38:24.448834 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.448733 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.448834 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.448775 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6gpdt\"" Jul 01 18:38:24.448952 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.448846 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.469116 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.469091 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr"] Jul 01 18:38:24.469211 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.469123 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn"] Jul 01 18:38:24.469211 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.469145 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-45t9c"] Jul 01 18:38:24.469211 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.469197 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" Jul 01 18:38:24.471223 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.471204 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k2qcc\"" Jul 01 18:38:24.527821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527797 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.527921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527838 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87f2696b-218e-4902-8a03-71a1c19b90c7-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.527921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527864 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/258d12c6-0758-4352-9dfc-b55aa56157e4-config\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.527921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527889 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd6xl\" (UniqueName: \"kubernetes.io/projected/87f2696b-218e-4902-8a03-71a1c19b90c7-kube-api-access-zd6xl\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.527921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527911 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rl4r\" (UniqueName: \"kubernetes.io/projected/258d12c6-0758-4352-9dfc-b55aa56157e4-kube-api-access-2rl4r\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:24.527919 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527937 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wss\" (UniqueName: \"kubernetes.io/projected/8983e4a6-bc80-4093-91b1-04f0f95be3c2-kube-api-access-t7wss\") pod \"network-check-source-56499946bc-45t9c\" (UID: \"8983e4a6-bc80-4093-91b1-04f0f95be3c2\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:24.527969 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:25.027956153 +0000 UTC m=+63.631116026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.527994 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82wgz\" (UniqueName: \"kubernetes.io/projected/ae045f82-c67a-48d1-9731-f3f999363168-kube-api-access-82wgz\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.528052 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87f2696b-218e-4902-8a03-71a1c19b90c7-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.528084 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/258d12c6-0758-4352-9dfc-b55aa56157e4-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.528134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.528132 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae045f82-c67a-48d1-9731-f3f999363168-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.528729 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.528712 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ae045f82-c67a-48d1-9731-f3f999363168-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.545661 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.545609 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82wgz\" (UniqueName: \"kubernetes.io/projected/ae045f82-c67a-48d1-9731-f3f999363168-kube-api-access-82wgz\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:24.628509 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87f2696b-218e-4902-8a03-71a1c19b90c7-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.628509 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628509 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/258d12c6-0758-4352-9dfc-b55aa56157e4-config\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.628700 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd6xl\" (UniqueName: \"kubernetes.io/projected/87f2696b-218e-4902-8a03-71a1c19b90c7-kube-api-access-zd6xl\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.628700 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rl4r\" (UniqueName: \"kubernetes.io/projected/258d12c6-0758-4352-9dfc-b55aa56157e4-kube-api-access-2rl4r\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.628700 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wss\" (UniqueName: \"kubernetes.io/projected/8983e4a6-bc80-4093-91b1-04f0f95be3c2-kube-api-access-t7wss\") pod \"network-check-source-56499946bc-45t9c\" (UID: \"8983e4a6-bc80-4093-91b1-04f0f95be3c2\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" Jul 01 18:38:24.628848 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628703 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87f2696b-218e-4902-8a03-71a1c19b90c7-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.628848 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.628730 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/258d12c6-0758-4352-9dfc-b55aa56157e4-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.629059 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.629039 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/258d12c6-0758-4352-9dfc-b55aa56157e4-config\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.629213 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.629197 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/87f2696b-218e-4902-8a03-71a1c19b90c7-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.630730 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.630711 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/87f2696b-218e-4902-8a03-71a1c19b90c7-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.630808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.630795 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/258d12c6-0758-4352-9dfc-b55aa56157e4-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.640428 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.640399 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rl4r\" (UniqueName: \"kubernetes.io/projected/258d12c6-0758-4352-9dfc-b55aa56157e4-kube-api-access-2rl4r\") pod \"service-ca-operator-5d7b7dc95c-wzlmn\" (UID: \"258d12c6-0758-4352-9dfc-b55aa56157e4\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.645868 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.645849 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd6xl\" (UniqueName: \"kubernetes.io/projected/87f2696b-218e-4902-8a03-71a1c19b90c7-kube-api-access-zd6xl\") pod \"kube-storage-version-migrator-operator-79964ff6f9-m55vr\" (UID: \"87f2696b-218e-4902-8a03-71a1c19b90c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.646430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.646410 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 18:38:24.656697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.656680 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 18:38:24.666489 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.666449 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wss\" (UniqueName: \"kubernetes.io/projected/8983e4a6-bc80-4093-91b1-04f0f95be3c2-kube-api-access-t7wss\") pod \"network-check-source-56499946bc-45t9c\" (UID: \"8983e4a6-bc80-4093-91b1-04f0f95be3c2\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" Jul 01 18:38:24.731839 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.731809 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" Jul 01 18:38:24.753966 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.753941 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" Jul 01 18:38:24.777118 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.777097 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" Jul 01 18:38:24.907877 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.907849 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn"] Jul 01 18:38:24.910501 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:24.910445 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258d12c6_0758_4352_9dfc_b55aa56157e4.slice/crio-775e8f66f35af8d5e8b925edba017d8807cc6e22288c77d16b1ae26061715e2c WatchSource:0}: Error finding container 775e8f66f35af8d5e8b925edba017d8807cc6e22288c77d16b1ae26061715e2c: Status 404 returned error can't find the container with id 775e8f66f35af8d5e8b925edba017d8807cc6e22288c77d16b1ae26061715e2c Jul 01 18:38:24.931992 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.931969 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr"] Jul 01 18:38:24.935307 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:24.935284 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f2696b_218e_4902_8a03_71a1c19b90c7.slice/crio-1bab827fad4365ce3a287048969cd4b6eec38a8873d28703ad8795272746eab6 WatchSource:0}: Error finding container 1bab827fad4365ce3a287048969cd4b6eec38a8873d28703ad8795272746eab6: Status 404 returned error can't find the container with id 1bab827fad4365ce3a287048969cd4b6eec38a8873d28703ad8795272746eab6 Jul 01 18:38:24.953315 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:24.953185 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-45t9c"] Jul 01 18:38:24.958441 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:24.958422 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8983e4a6_bc80_4093_91b1_04f0f95be3c2.slice/crio-4c669103d0f88ff82f05f63d92f925827fb366d504e8c92d1d7bfe64b0add545 WatchSource:0}: Error finding container 4c669103d0f88ff82f05f63d92f925827fb366d504e8c92d1d7bfe64b0add545: Status 404 returned error can't find the container with id 4c669103d0f88ff82f05f63d92f925827fb366d504e8c92d1d7bfe64b0add545 Jul 01 18:38:25.031955 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:25.031930 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:25.032089 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:25.032070 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:25.032139 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:25.032129 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:26.032114467 +0000 UTC m=+64.635274342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:25.096209 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:25.096141 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" event={"ID":"87f2696b-218e-4902-8a03-71a1c19b90c7","Type":"ContainerStarted","Data":"1bab827fad4365ce3a287048969cd4b6eec38a8873d28703ad8795272746eab6"} Jul 01 18:38:25.097214 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:25.097178 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" event={"ID":"8983e4a6-bc80-4093-91b1-04f0f95be3c2","Type":"ContainerStarted","Data":"4c669103d0f88ff82f05f63d92f925827fb366d504e8c92d1d7bfe64b0add545"} Jul 01 18:38:25.098152 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:25.098132 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" event={"ID":"258d12c6-0758-4352-9dfc-b55aa56157e4","Type":"ContainerStarted","Data":"775e8f66f35af8d5e8b925edba017d8807cc6e22288c77d16b1ae26061715e2c"} Jul 01 18:38:26.038687 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.038622 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:26.039140 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:26.038804 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:26.039140 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:26.038935 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:28.038909695 +0000 UTC m=+66.642069590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:26.541862 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.541827 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:38:26.546248 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.546027 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 18:38:26.552884 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:26.552856 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 18:38:26.553012 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:26.552930 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs podName:2c9cac5a-22da-46f2-a514-049a7075ea4a nodeName:}" failed. No retries permitted until 2026-07-01 18:39:30.552909365 +0000 UTC m=+129.156069239 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs") pod "network-metrics-daemon-6hvdk" (UID: "2c9cac5a-22da-46f2-a514-049a7075ea4a") : secret "metrics-daemon-secret" not found Jul 01 18:38:26.642980 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.642513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:38:26.642980 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.642604 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:38:26.645892 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.645867 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndlpn\" (UniqueName: \"kubernetes.io/projected/1bdf0907-9f42-406f-a37e-49e5cc80e144-kube-api-access-ndlpn\") pod \"network-check-target-twz9h\" (UID: \"1bdf0907-9f42-406f-a37e-49e5cc80e144\") " pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:38:26.646091 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.646066 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 18:38:26.655512 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.655486 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7bae8d16-a7ec-416b-874a-385f16b4b780-original-pull-secret\") pod \"global-pull-secret-syncer-jkrhv\" (UID: \"7bae8d16-a7ec-416b-874a-385f16b4b780\") " pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:38:26.730273 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.730234 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrhv" Jul 01 18:38:26.741036 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.741009 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lwv7s\"" Jul 01 18:38:26.749176 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:26.749127 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:38:27.257896 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.257863 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ljtr7"] Jul 01 18:38:27.260940 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.260915 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.263988 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.263965 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 18:38:27.263988 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.263965 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 18:38:27.264187 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.264009 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 18:38:27.264187 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.263972 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q4zk9\"" Jul 01 18:38:27.264653 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.264630 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 18:38:27.279576 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.279549 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ljtr7"] Jul 01 18:38:27.347970 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.347940 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.348141 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.347998 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-crio-socket\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.348141 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.348027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.348141 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.348092 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-data-volume\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.348141 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.348120 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m568j\" (UniqueName: \"kubernetes.io/projected/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-api-access-m568j\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.448591 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448561 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.448752 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448608 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-crio-socket\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.448752 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.448725 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.448752 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448741 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.448911 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.448802 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:27.948776679 +0000 UTC m=+66.551936566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.448911 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448852 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-data-volume\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.448911 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448886 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m568j\" (UniqueName: \"kubernetes.io/projected/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-api-access-m568j\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.449065 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.448888 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:27.449065 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.448851 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-crio-socket\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.449065 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.448981 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:27.948969109 +0000 UTC m=+66.552128984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : configmap "kube-rbac-proxy" not found Jul 01 18:38:27.449212 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.449141 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-data-volume\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.462962 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.462939 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m568j\" (UniqueName: \"kubernetes.io/projected/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-api-access-m568j\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.953740 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.953706 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.953965 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:27.953760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:27.953965 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.953856 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:27.953965 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.953881 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:27.953965 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.953923 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:28.953909149 +0000 UTC m=+67.557069020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : configmap "kube-rbac-proxy" not found Jul 01 18:38:27.953965 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:27.953949 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:28.953929561 +0000 UTC m=+67.557089433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:28.054571 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:28.054536 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:28.054746 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.054694 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:28.054806 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.054762 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:32.054746006 +0000 UTC m=+70.657905892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:28.499783 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:28.499745 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-twz9h"] Jul 01 18:38:28.504369 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:28.504335 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bdf0907_9f42_406f_a37e_49e5cc80e144.slice/crio-123b27f8ba0487b3aa3a131a00e55000a30cb5ded23d7444a976e0bec753502d WatchSource:0}: Error finding container 123b27f8ba0487b3aa3a131a00e55000a30cb5ded23d7444a976e0bec753502d: Status 404 returned error can't find the container with id 123b27f8ba0487b3aa3a131a00e55000a30cb5ded23d7444a976e0bec753502d Jul 01 18:38:28.520301 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:28.518291 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkrhv"] Jul 01 18:38:28.527660 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:28.527634 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bae8d16_a7ec_416b_874a_385f16b4b780.slice/crio-bf303d7e033f4bbd9d50f81d5e703c99cb0c85c7c0d9e0cc616bd434b0d3e816 WatchSource:0}: Error finding container bf303d7e033f4bbd9d50f81d5e703c99cb0c85c7c0d9e0cc616bd434b0d3e816: Status 404 returned error can't find the container with id bf303d7e033f4bbd9d50f81d5e703c99cb0c85c7c0d9e0cc616bd434b0d3e816 Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:28.962073 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:28.962130 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.962230 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.962288 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:30.962270365 +0000 UTC m=+69.565430239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : configmap "kube-rbac-proxy" not found Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.962379 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:28.962493 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:28.962426 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:30.9624125 +0000 UTC m=+69.565572388 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:29.113549 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.113257 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" event={"ID":"8983e4a6-bc80-4093-91b1-04f0f95be3c2","Type":"ContainerStarted","Data":"5b3b3c003f32960ae5fea7dc22a8854897b7aa6b21491fc71d749cb68701e221"} Jul 01 18:38:29.115139 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.115111 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" event={"ID":"258d12c6-0758-4352-9dfc-b55aa56157e4","Type":"ContainerStarted","Data":"60a86e882c8739bbb95b07dcf7db30a0fc74c3b34845cfe5f2514f0a7a38088f"} Jul 01 18:38:29.116708 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.116676 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkrhv" event={"ID":"7bae8d16-a7ec-416b-874a-385f16b4b780","Type":"ContainerStarted","Data":"bf303d7e033f4bbd9d50f81d5e703c99cb0c85c7c0d9e0cc616bd434b0d3e816"} Jul 01 18:38:29.118442 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.118370 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-twz9h" event={"ID":"1bdf0907-9f42-406f-a37e-49e5cc80e144","Type":"ContainerStarted","Data":"e112d67ae24175e0956fb419c18b2944dd51674a03e39a390191c6023a5827e3"} Jul 01 18:38:29.118442 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.118412 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-twz9h" event={"ID":"1bdf0907-9f42-406f-a37e-49e5cc80e144","Type":"ContainerStarted","Data":"123b27f8ba0487b3aa3a131a00e55000a30cb5ded23d7444a976e0bec753502d"} Jul 01 18:38:29.118697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.118642 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:38:29.120328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.120267 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" event={"ID":"87f2696b-218e-4902-8a03-71a1c19b90c7","Type":"ContainerStarted","Data":"b410e40f106d3fa373c40c1a4ef8450e1f4a81b28c2126b282a775662ece873b"} Jul 01 18:38:29.139700 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.139646 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-45t9c" podStartSLOduration=1.743354558 podStartE2EDuration="5.139629933s" totalImagesPullingTime="3.396275375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:29.138303756 +0000 UTC m=+67.741463650" watchObservedRunningTime="2026-07-01 18:38:29.139629933 +0000 UTC m=+67.742789852" Jul 01 18:38:29.171780 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.171234 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" podStartSLOduration=1.7611528029999999 podStartE2EDuration="5.171218107s" totalImagesPullingTime="3.410065304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:29.170361096 +0000 UTC m=+67.773520991" watchObservedRunningTime="2026-07-01 18:38:29.171218107 +0000 UTC m=+67.774378001" Jul 01 18:38:29.201496 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.201390 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" podStartSLOduration=1.759141542 podStartE2EDuration="5.201373146s" totalImagesPullingTime="3.442231604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:29.20134404 +0000 UTC m=+67.804503932" watchObservedRunningTime="2026-07-01 18:38:29.201373146 +0000 UTC m=+67.804533041" Jul 01 18:38:29.404829 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.404799 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nb24j_1d5ef2ef-3ddb-46b8-8f60-aaad32410974/dns-node-resolver/0.log" Jul 01 18:38:29.816257 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:29.816228 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5f79x_cf08c72b-7225-4ad1-9989-82b43ce8337a/node-ca/0.log" Jul 01 18:38:30.578640 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:30.578561 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:38:30.578915 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.578743 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 18:38:30.578915 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.578825 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls podName:fce2c7b4-6610-44c0-882d-d816c734aa6e nodeName:}" failed. No retries permitted until 2026-07-01 18:39:02.578804697 +0000 UTC m=+101.181964571 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls") pod "dns-default-zr9jd" (UID: "fce2c7b4-6610-44c0-882d-d816c734aa6e") : secret "dns-default-metrics-tls" not found Jul 01 18:38:30.804496 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:30.804447 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b72ng_e455c3a6-5452-4971-8b72-072592a96d92/serve-healthcheck-canary/0.log" Jul 01 18:38:30.981364 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:30.981284 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:30.981364 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:30.981350 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:30.981863 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.981473 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:30.981863 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.981479 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:30.981863 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.981550 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:34.981529049 +0000 UTC m=+73.584688936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : configmap "kube-rbac-proxy" not found Jul 01 18:38:30.981863 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:30.981585 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:34.981564636 +0000 UTC m=+73.584724515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:31.173352 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.173292 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-twz9h" podStartSLOduration=69.173274611 podStartE2EDuration="1m9.173274611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:29.230037403 +0000 UTC m=+67.833197296" watchObservedRunningTime="2026-07-01 18:38:31.173274611 +0000 UTC m=+69.776434486" Jul 01 18:38:31.174817 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.174765 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:38:31.198792 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.198763 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:38:31.199096 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.199064 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.203695 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.202284 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 18:38:31.203695 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.202590 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 18:38:31.203695 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.202859 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 18:38:31.203695 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.203011 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x2fbb\"" Jul 01 18:38:31.208490 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.208469 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 18:38:31.284157 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284315 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284231 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284315 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284267 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284427 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284370 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284500 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284554 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284512 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284606 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284550 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjrt\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.284695 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.284672 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386112 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386079 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386259 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386335 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386304 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386388 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386335 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjrt\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386388 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386381 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.386398 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.386415 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc666f464-qmqsp: secret "image-registry-tls" not found Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386416 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386488 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.386505 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls podName:dc023fe8-4e56-477d-99a1-9fdbe29174d0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:31.88648553 +0000 UTC m=+70.489645416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls") pod "image-registry-5bc666f464-qmqsp" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0") : secret "image-registry-tls" not found Jul 01 18:38:31.386536 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.386837 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.386556 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.387159 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.387137 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.387567 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.387545 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.389134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.389112 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.389265 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.389241 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.406025 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.406000 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.406151 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.406040 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjrt\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.889767 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:31.889731 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:31.889955 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.889905 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:31.889955 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.889926 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc666f464-qmqsp: secret "image-registry-tls" not found Jul 01 18:38:31.890057 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:31.890011 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls podName:dc023fe8-4e56-477d-99a1-9fdbe29174d0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:32.889989533 +0000 UTC m=+71.493149427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls") pod "image-registry-5bc666f464-qmqsp" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0") : secret "image-registry-tls" not found Jul 01 18:38:32.091531 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:32.091483 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:32.092015 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:32.091655 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:32.092015 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:32.091745 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:40.091723787 +0000 UTC m=+78.694883659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:32.899436 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:32.899406 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:32.899598 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:32.899545 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:32.899598 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:32.899559 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc666f464-qmqsp: secret "image-registry-tls" not found Jul 01 18:38:32.899670 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:32.899610 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls podName:dc023fe8-4e56-477d-99a1-9fdbe29174d0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:34.899592493 +0000 UTC m=+73.502752369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls") pod "image-registry-5bc666f464-qmqsp" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0") : secret "image-registry-tls" not found Jul 01 18:38:33.135822 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:33.135788 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkrhv" event={"ID":"7bae8d16-a7ec-416b-874a-385f16b4b780","Type":"ContainerStarted","Data":"ff63025120bda5a9900e576d9064b824affbee2a619625b9e4f7112c4332d4e9"} Jul 01 18:38:33.164151 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:33.164100 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jkrhv" podStartSLOduration=66.99106671 podStartE2EDuration="1m11.164084965s" totalImagesPullingTime="4.173018255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:33.162535059 +0000 UTC m=+71.765694953" watchObservedRunningTime="2026-07-01 18:38:33.164084965 +0000 UTC m=+71.767244859" Jul 01 18:38:34.915247 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:34.915189 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:34.915682 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:34.915367 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:34.915682 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:34.915386 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc666f464-qmqsp: secret "image-registry-tls" not found Jul 01 18:38:34.915682 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:34.915477 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls podName:dc023fe8-4e56-477d-99a1-9fdbe29174d0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:38.915441595 +0000 UTC m=+77.518601466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls") pod "image-registry-5bc666f464-qmqsp" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0") : secret "image-registry-tls" not found Jul 01 18:38:35.016389 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:35.016352 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:35.016578 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:35.016514 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 18:38:35.016578 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:35.016554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:35.016700 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:35.016601 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:43.016579257 +0000 UTC m=+81.619739133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : configmap "kube-rbac-proxy" not found Jul 01 18:38:35.016700 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:35.016668 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:35.016804 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:35.016714 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:43.016700867 +0000 UTC m=+81.619860752 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:38.944239 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:38.944203 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:38.944683 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:38.944348 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 18:38:38.944683 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:38.944369 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bc666f464-qmqsp: secret "image-registry-tls" not found Jul 01 18:38:38.944683 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:38.944423 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls podName:dc023fe8-4e56-477d-99a1-9fdbe29174d0 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:46.944406874 +0000 UTC m=+85.547566746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls") pod "image-registry-5bc666f464-qmqsp" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0") : secret "image-registry-tls" not found Jul 01 18:38:40.155281 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:40.155244 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:40.155771 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:40.155364 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:40.155771 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:40.155413 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls podName:ae045f82-c67a-48d1-9731-f3f999363168 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:56.155400969 +0000 UTC m=+94.758560841 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-n5g6z" (UID: "ae045f82-c67a-48d1-9731-f3f999363168") : secret "cluster-monitoring-operator-tls" not found Jul 01 18:38:43.079366 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:43.079320 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:43.079774 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:43.079385 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:43.079774 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:43.079493 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 18:38:43.079774 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:43.079567 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls podName:d3ac98d7-225d-42ff-bd2b-4ad4f43e3090 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:59.079551885 +0000 UTC m=+97.682711761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ljtr7" (UID: "d3ac98d7-225d-42ff-bd2b-4ad4f43e3090") : secret "insights-runtime-extractor-tls" not found Jul 01 18:38:43.079920 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:43.079903 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:47.014715 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:47.014592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:47.016927 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:47.016906 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"image-registry-5bc666f464-qmqsp\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:47.113830 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:47.113790 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:47.252415 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:47.252381 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:38:47.255191 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:47.255164 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc023fe8_4e56_477d_99a1_9fdbe29174d0.slice/crio-1ae40a6726d4923e40c44db764621ecf853b779f1d672eb47da70cab0f9bb8d0 WatchSource:0}: Error finding container 1ae40a6726d4923e40c44db764621ecf853b779f1d672eb47da70cab0f9bb8d0: Status 404 returned error can't find the container with id 1ae40a6726d4923e40c44db764621ecf853b779f1d672eb47da70cab0f9bb8d0 Jul 01 18:38:48.179337 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:48.179303 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" event={"ID":"dc023fe8-4e56-477d-99a1-9fdbe29174d0","Type":"ContainerStarted","Data":"dd1864423d472348d69f71d64acf48b62a806deaa040013f0483266baedd5481"} Jul 01 18:38:48.179735 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:48.179353 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" event={"ID":"dc023fe8-4e56-477d-99a1-9fdbe29174d0","Type":"ContainerStarted","Data":"1ae40a6726d4923e40c44db764621ecf853b779f1d672eb47da70cab0f9bb8d0"} Jul 01 18:38:48.179735 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:48.179422 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:38:48.220645 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:48.220600 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" podStartSLOduration=17.220586782 podStartE2EDuration="17.220586782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:48.220499714 +0000 UTC m=+86.823659608" watchObservedRunningTime="2026-07-01 18:38:48.220586782 +0000 UTC m=+86.823746675" Jul 01 18:38:55.273623 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.273592 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v"] Jul 01 18:38:55.276696 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.276678 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.280470 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.280424 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 18:38:55.280599 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.280520 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 01 18:38:55.280599 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.280558 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 18:38:55.280828 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.280814 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 18:38:55.303746 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.303722 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v"] Jul 01 18:38:55.374990 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.374957 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7dlv\" (UniqueName: \"kubernetes.io/projected/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-kube-api-access-w7dlv\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.375192 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.375029 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-klusterlet-config\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.375192 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.375138 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-tmp\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.401914 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.401878 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:38:55.457580 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.457548 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d65c54b89-nfd6t"] Jul 01 18:38:55.459522 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.459480 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.475594 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.475574 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-tmp\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.475701 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.475617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7dlv\" (UniqueName: \"kubernetes.io/projected/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-kube-api-access-w7dlv\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.475701 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.475651 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-klusterlet-config\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.475998 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.475975 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-tmp\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.478024 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.478002 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-klusterlet-config\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.489167 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.489147 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d65c54b89-nfd6t"] Jul 01 18:38:55.510345 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.510318 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7dlv\" (UniqueName: \"kubernetes.io/projected/59c1c1be-75c0-48fa-a8e7-5a97ed8c6787-kube-api-access-w7dlv\") pod \"klusterlet-addon-workmgr-8b955f794-z8h7v\" (UID: \"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.576495 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576411 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-trusted-ca\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576495 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576442 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-certificates\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576637 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576505 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-image-registry-private-configuration\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576637 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576597 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-tls\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576637 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576613 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f955947e-1e6b-4b74-936f-adb2d918f9f3-ca-trust-extracted\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576637 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576629 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-installation-pull-secrets\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576761 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576647 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p78m\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-kube-api-access-6p78m\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.576761 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.576670 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-bound-sa-token\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.585425 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.585403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:38:55.678141 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678113 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-trusted-ca\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678302 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-certificates\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678302 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678170 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-image-registry-private-configuration\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678405 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-tls\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678489 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678432 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f955947e-1e6b-4b74-936f-adb2d918f9f3-ca-trust-extracted\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678543 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-installation-pull-secrets\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678543 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678521 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p78m\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-kube-api-access-6p78m\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.678650 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.678550 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-bound-sa-token\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.679066 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.679006 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f955947e-1e6b-4b74-936f-adb2d918f9f3-ca-trust-extracted\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.679176 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.679074 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-certificates\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.679396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.679376 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f955947e-1e6b-4b74-936f-adb2d918f9f3-trusted-ca\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.680821 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.680801 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-registry-tls\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.680936 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.680884 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-image-registry-private-configuration\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.681030 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.681011 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f955947e-1e6b-4b74-936f-adb2d918f9f3-installation-pull-secrets\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.698043 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.698020 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-bound-sa-token\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.698427 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.698411 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p78m\" (UniqueName: \"kubernetes.io/projected/f955947e-1e6b-4b74-936f-adb2d918f9f3-kube-api-access-6p78m\") pod \"image-registry-d65c54b89-nfd6t\" (UID: \"f955947e-1e6b-4b74-936f-adb2d918f9f3\") " pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.729808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.729784 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v"] Jul 01 18:38:55.732677 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:55.732659 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c1c1be_75c0_48fa_a8e7_5a97ed8c6787.slice/crio-b20189761249517f236d838e2de3a19db4e2b492aaf7e8fd42c6ec9dc7bf1bc7 WatchSource:0}: Error finding container b20189761249517f236d838e2de3a19db4e2b492aaf7e8fd42c6ec9dc7bf1bc7: Status 404 returned error can't find the container with id b20189761249517f236d838e2de3a19db4e2b492aaf7e8fd42c6ec9dc7bf1bc7 Jul 01 18:38:55.768357 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.768337 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:55.901022 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:55.900993 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d65c54b89-nfd6t"] Jul 01 18:38:55.904205 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:55.904179 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf955947e_1e6b_4b74_936f_adb2d918f9f3.slice/crio-df15f4063e006982ed55c3bbda568b4748af5b1b0cadd71215e0e8dbb79a8442 WatchSource:0}: Error finding container df15f4063e006982ed55c3bbda568b4748af5b1b0cadd71215e0e8dbb79a8442: Status 404 returned error can't find the container with id df15f4063e006982ed55c3bbda568b4748af5b1b0cadd71215e0e8dbb79a8442 Jul 01 18:38:56.182800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.182722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:56.184971 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.184946 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae045f82-c67a-48d1-9731-f3f999363168-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-n5g6z\" (UID: \"ae045f82-c67a-48d1-9731-f3f999363168\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:56.206145 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.206117 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" event={"ID":"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787","Type":"ContainerStarted","Data":"b20189761249517f236d838e2de3a19db4e2b492aaf7e8fd42c6ec9dc7bf1bc7"} Jul 01 18:38:56.207604 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.207583 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" event={"ID":"f955947e-1e6b-4b74-936f-adb2d918f9f3","Type":"ContainerStarted","Data":"8956a6f19980340d2b370cda9cb1fc2110e449ee271acab5c3ffc11d42eaea7d"} Jul 01 18:38:56.207682 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.207618 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" event={"ID":"f955947e-1e6b-4b74-936f-adb2d918f9f3","Type":"ContainerStarted","Data":"df15f4063e006982ed55c3bbda568b4748af5b1b0cadd71215e0e8dbb79a8442"} Jul 01 18:38:56.207739 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.207723 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:38:56.435865 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.435791 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" Jul 01 18:38:56.571761 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.571667 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" podStartSLOduration=1.571643361 podStartE2EDuration="1.571643361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:56.241940246 +0000 UTC m=+94.845100152" watchObservedRunningTime="2026-07-01 18:38:56.571643361 +0000 UTC m=+95.174803256" Jul 01 18:38:56.572304 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:56.572284 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z"] Jul 01 18:38:56.575406 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:56.575379 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae045f82_c67a_48d1_9731_f3f999363168.slice/crio-bcd4375d894a1d96673f5454fb777dd438d2bcb79be25332ad093d9998626ab5 WatchSource:0}: Error finding container bcd4375d894a1d96673f5454fb777dd438d2bcb79be25332ad093d9998626ab5: Status 404 returned error can't find the container with id bcd4375d894a1d96673f5454fb777dd438d2bcb79be25332ad093d9998626ab5 Jul 01 18:38:57.212301 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:57.212263 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" event={"ID":"ae045f82-c67a-48d1-9731-f3f999363168","Type":"ContainerStarted","Data":"bcd4375d894a1d96673f5454fb777dd438d2bcb79be25332ad093d9998626ab5"} Jul 01 18:38:58.221627 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.221585 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" event={"ID":"ae045f82-c67a-48d1-9731-f3f999363168","Type":"ContainerStarted","Data":"71b75d81bbb7f3b7c6fd815b23a1f2088038ebb4493445025cfe88dfc204a48d"} Jul 01 18:38:58.243601 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.243542 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-n5g6z" podStartSLOduration=32.794555557 podStartE2EDuration="34.243520216s" totalImagesPullingTime="1.448964659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:38:58.24331432 +0000 UTC m=+96.846474216" watchObservedRunningTime="2026-07-01 18:38:58.243520216 +0000 UTC m=+96.846680112" Jul 01 18:38:58.625102 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.625073 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:38:58.626913 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.626890 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.629405 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.629383 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 18:38:58.629575 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.629423 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 18:38:58.629575 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.629424 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 18:38:58.629575 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.629428 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-47s2p\"" Jul 01 18:38:58.630051 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.630030 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 18:38:58.630155 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.630066 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 18:38:58.630155 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.630067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 18:38:58.630155 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.630092 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 18:38:58.647005 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.646988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:38:58.689714 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.689691 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj"] Jul 01 18:38:58.691979 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.691963 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:58.695047 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.695029 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 01 18:38:58.695150 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.695060 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6tcvp\"" Jul 01 18:38:58.700848 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700830 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.700917 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700875 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rbsd\" (UniqueName: \"kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.700967 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700915 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.700967 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700944 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.700967 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700959 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.701083 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.700986 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.706907 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.706890 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj"] Jul 01 18:38:58.802353 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802319 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802505 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802382 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802505 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802411 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802505 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802474 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802523 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802572 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rbsd\" (UniqueName: \"kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.802659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.802606 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-rb6vj\" (UID: \"edf1f520-483f-4347-a2ee-1fb9706c9138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:58.803332 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.803311 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.803573 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.803547 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.803975 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.803933 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.805118 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.805072 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.805532 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.805513 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.815492 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.815448 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rbsd\" (UniqueName: \"kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd\") pod \"console-85cdf66d46-m5dgv\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:58.903829 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.903746 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-rb6vj\" (UID: \"edf1f520-483f-4347-a2ee-1fb9706c9138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:58.903965 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:58.903910 2554 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 01 18:38:58.904025 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:38:58.904010 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates podName:edf1f520-483f-4347-a2ee-1fb9706c9138 nodeName:}" failed. No retries permitted until 2026-07-01 18:38:59.40398891 +0000 UTC m=+98.007148805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates") pod "prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" (UID: "edf1f520-483f-4347-a2ee-1fb9706c9138") : secret "prometheus-operator-admission-webhook-tls" not found Jul 01 18:38:58.937407 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:58.937382 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:38:59.087110 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.087082 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:38:59.089472 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:59.089423 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ea25d9_886e_4346_a42a_828fc4944ad4.slice/crio-2d30cb141006a848792249d9059f7d0c37d7b6bff23f6d8117f7d1565ffa15c7 WatchSource:0}: Error finding container 2d30cb141006a848792249d9059f7d0c37d7b6bff23f6d8117f7d1565ffa15c7: Status 404 returned error can't find the container with id 2d30cb141006a848792249d9059f7d0c37d7b6bff23f6d8117f7d1565ffa15c7 Jul 01 18:38:59.105321 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.105293 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:59.107738 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.107714 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d3ac98d7-225d-42ff-bd2b-4ad4f43e3090-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ljtr7\" (UID: \"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090\") " pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:59.227011 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.226930 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdf66d46-m5dgv" event={"ID":"b7ea25d9-886e-4346-a42a-828fc4944ad4","Type":"ContainerStarted","Data":"2d30cb141006a848792249d9059f7d0c37d7b6bff23f6d8117f7d1565ffa15c7"} Jul 01 18:38:59.372153 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.372115 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ljtr7" Jul 01 18:38:59.408343 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.408223 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-rb6vj\" (UID: \"edf1f520-483f-4347-a2ee-1fb9706c9138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:59.411932 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.411883 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/edf1f520-483f-4347-a2ee-1fb9706c9138-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-rb6vj\" (UID: \"edf1f520-483f-4347-a2ee-1fb9706c9138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:59.539420 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.539326 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ljtr7"] Jul 01 18:38:59.542338 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:59.542305 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ac98d7_225d_42ff_bd2b_4ad4f43e3090.slice/crio-470463f328db4d320d67dbb92ba336a0d0578683dadd7798a42a1f4436dc632a WatchSource:0}: Error finding container 470463f328db4d320d67dbb92ba336a0d0578683dadd7798a42a1f4436dc632a: Status 404 returned error can't find the container with id 470463f328db4d320d67dbb92ba336a0d0578683dadd7798a42a1f4436dc632a Jul 01 18:38:59.601703 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.601670 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:38:59.782471 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:38:59.782377 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj"] Jul 01 18:38:59.786795 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:38:59.786732 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf1f520_483f_4347_a2ee_1fb9706c9138.slice/crio-db48cfb4f21b933f56c5c00f525e4316e7d985fb9eb7afcb58d635ccc0803b94 WatchSource:0}: Error finding container db48cfb4f21b933f56c5c00f525e4316e7d985fb9eb7afcb58d635ccc0803b94: Status 404 returned error can't find the container with id db48cfb4f21b933f56c5c00f525e4316e7d985fb9eb7afcb58d635ccc0803b94 Jul 01 18:39:00.127668 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:00.126835 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-twz9h" Jul 01 18:39:00.235780 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:00.235681 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" event={"ID":"edf1f520-483f-4347-a2ee-1fb9706c9138","Type":"ContainerStarted","Data":"db48cfb4f21b933f56c5c00f525e4316e7d985fb9eb7afcb58d635ccc0803b94"} Jul 01 18:39:00.238226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:00.238193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ljtr7" event={"ID":"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090","Type":"ContainerStarted","Data":"470463f328db4d320d67dbb92ba336a0d0578683dadd7798a42a1f4436dc632a"} Jul 01 18:39:02.639598 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:02.639556 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:39:02.642559 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:02.642504 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fce2c7b4-6610-44c0-882d-d816c734aa6e-metrics-tls\") pod \"dns-default-zr9jd\" (UID: \"fce2c7b4-6610-44c0-882d-d816c734aa6e\") " pod="openshift-dns/dns-default-zr9jd" Jul 01 18:39:02.679354 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:02.679317 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7sn7m\"" Jul 01 18:39:02.686628 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:02.686594 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zr9jd" Jul 01 18:39:05.408249 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:05.408221 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:39:06.762413 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:06.762383 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zr9jd"] Jul 01 18:39:06.765807 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:06.765784 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce2c7b4_6610_44c0_882d_d816c734aa6e.slice/crio-2a307d91abede6e79cd49b135e92c3fa8ec04c358bef25eca509e60031164a69 WatchSource:0}: Error finding container 2a307d91abede6e79cd49b135e92c3fa8ec04c358bef25eca509e60031164a69: Status 404 returned error can't find the container with id 2a307d91abede6e79cd49b135e92c3fa8ec04c358bef25eca509e60031164a69 Jul 01 18:39:07.273857 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.273393 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdf66d46-m5dgv" event={"ID":"b7ea25d9-886e-4346-a42a-828fc4944ad4","Type":"ContainerStarted","Data":"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721"} Jul 01 18:39:07.276131 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.276067 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr9jd" event={"ID":"fce2c7b4-6610-44c0-882d-d816c734aa6e","Type":"ContainerStarted","Data":"2a307d91abede6e79cd49b135e92c3fa8ec04c358bef25eca509e60031164a69"} Jul 01 18:39:07.277630 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.277572 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" event={"ID":"59c1c1be-75c0-48fa-a8e7-5a97ed8c6787","Type":"ContainerStarted","Data":"8f5c5bf4fb9f5a302f358d4620c98c67bd5ddae67020accbfc494bfa1e1f7e6a"} Jul 01 18:39:07.277872 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.277776 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:39:07.279325 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.279296 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" event={"ID":"edf1f520-483f-4347-a2ee-1fb9706c9138","Type":"ContainerStarted","Data":"9ae325002b60c7b5f657499ad217ce2834d63c0fed48318db1ec240e6248711c"} Jul 01 18:39:07.279593 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.279441 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:39:07.279673 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.279639 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" Jul 01 18:39:07.281507 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.281487 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ljtr7" event={"ID":"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090","Type":"ContainerStarted","Data":"39b482a5f31b200186b07243f2e426228ec90f8658ae2bdc5c9e2450ef90bafa"} Jul 01 18:39:07.281606 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.281532 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ljtr7" event={"ID":"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090","Type":"ContainerStarted","Data":"892f3c0f5c22ebf11317a5fc5dab2a9a5e038be6c478154656df14fff5b8df2a"} Jul 01 18:39:07.284722 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.284703 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" Jul 01 18:39:07.298462 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.298409 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85cdf66d46-m5dgv" podStartSLOduration=1.7965138889999999 podStartE2EDuration="9.298392088s" totalImagesPullingTime="7.501878199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:07.297519627 +0000 UTC m=+105.900679524" watchObservedRunningTime="2026-07-01 18:39:07.298392088 +0000 UTC m=+105.901551983" Jul 01 18:39:07.319495 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.319431 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-rb6vj" podStartSLOduration=2.5091295970000003 podStartE2EDuration="9.31939978s" totalImagesPullingTime="6.810270183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:07.318975112 +0000 UTC m=+105.922135007" watchObservedRunningTime="2026-07-01 18:39:07.31939978 +0000 UTC m=+105.922559676" Jul 01 18:39:07.345193 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.345138 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8b955f794-z8h7v" podStartSLOduration=2.075953872 podStartE2EDuration="12.345118107s" totalImagesPullingTime="10.269164235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:07.341994964 +0000 UTC m=+105.945154857" watchObservedRunningTime="2026-07-01 18:39:07.345118107 +0000 UTC m=+105.948278002" Jul 01 18:39:07.873155 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.873126 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-f6xlc"] Jul 01 18:39:07.875677 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.875654 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:07.878954 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.878929 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 18:39:07.880769 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.880748 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 18:39:07.881557 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.881397 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 18:39:07.881557 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.881476 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-x6kdt\"" Jul 01 18:39:07.894739 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.894716 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-f6xlc"] Jul 01 18:39:07.986544 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.986503 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab11438-9d3e-4878-8984-47e11923a323-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:07.986713 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.986562 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:07.986713 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.986585 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjs6\" (UniqueName: \"kubernetes.io/projected/bab11438-9d3e-4878-8984-47e11923a323-kube-api-access-8mjs6\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:07.986713 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:07.986609 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.087862 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.087820 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab11438-9d3e-4878-8984-47e11923a323-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.088045 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.087876 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.088045 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.087903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjs6\" (UniqueName: \"kubernetes.io/projected/bab11438-9d3e-4878-8984-47e11923a323-kube-api-access-8mjs6\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.088045 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.087936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.089201 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.089176 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab11438-9d3e-4878-8984-47e11923a323-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.091152 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.091104 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.091271 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.091190 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/bab11438-9d3e-4878-8984-47e11923a323-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.103616 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.103570 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjs6\" (UniqueName: \"kubernetes.io/projected/bab11438-9d3e-4878-8984-47e11923a323-kube-api-access-8mjs6\") pod \"prometheus-operator-7b7d896497-f6xlc\" (UID: \"bab11438-9d3e-4878-8984-47e11923a323\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.188536 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.188440 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" Jul 01 18:39:08.373620 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.373599 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-f6xlc"] Jul 01 18:39:08.377463 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:08.377412 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab11438_9d3e_4878_8984_47e11923a323.slice/crio-83541f8b6e4b15eeea56f7b355c8d3dbfd46a75a5c879f327b08f1dcf30df6e9 WatchSource:0}: Error finding container 83541f8b6e4b15eeea56f7b355c8d3dbfd46a75a5c879f327b08f1dcf30df6e9: Status 404 returned error can't find the container with id 83541f8b6e4b15eeea56f7b355c8d3dbfd46a75a5c879f327b08f1dcf30df6e9 Jul 01 18:39:08.938025 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.937990 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:39:08.938395 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.938056 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:39:08.943372 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:08.943350 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:39:09.294984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:09.294947 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" event={"ID":"bab11438-9d3e-4878-8984-47e11923a323","Type":"ContainerStarted","Data":"83541f8b6e4b15eeea56f7b355c8d3dbfd46a75a5c879f327b08f1dcf30df6e9"} Jul 01 18:39:09.297041 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:09.297007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr9jd" event={"ID":"fce2c7b4-6610-44c0-882d-d816c734aa6e","Type":"ContainerStarted","Data":"0e181ca965e6274ab36987d6ac52884e1fe3ed24c43ac684bf1e739b2906b49d"} Jul 01 18:39:09.297183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:09.297066 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr9jd" event={"ID":"fce2c7b4-6610-44c0-882d-d816c734aa6e","Type":"ContainerStarted","Data":"4ab3cbfaff2a4a768b080f8b15115741a98321aabe4a217ac23f31d6dd71f9bc"} Jul 01 18:39:09.301707 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:09.301686 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:39:09.323657 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:09.323599 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zr9jd" podStartSLOduration=69.838800841 podStartE2EDuration="1m11.323578772s" totalImagesPullingTime="1.484777931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:09.321559859 +0000 UTC m=+107.924719754" watchObservedRunningTime="2026-07-01 18:39:09.323578772 +0000 UTC m=+107.926738691" Jul 01 18:39:10.302544 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:10.302426 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ljtr7" event={"ID":"d3ac98d7-225d-42ff-bd2b-4ad4f43e3090","Type":"ContainerStarted","Data":"6b5ed0e607422219f56bc450056fc9f16bea28d14b602e236bd7ed2c94e9155d"} Jul 01 18:39:10.303022 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:10.302853 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zr9jd" Jul 01 18:39:10.331646 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:10.331596 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ljtr7" podStartSLOduration=33.232168769 podStartE2EDuration="43.331580201s" totalImagesPullingTime="10.099411432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:38:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:10.329542432 +0000 UTC m=+108.932702327" watchObservedRunningTime="2026-07-01 18:39:10.331580201 +0000 UTC m=+108.934740095" Jul 01 18:39:11.307830 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:11.307795 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" event={"ID":"bab11438-9d3e-4878-8984-47e11923a323","Type":"ContainerStarted","Data":"187c2e5f37cf4092de4e16795c10f3eb5100c5c68901d5625376038acecae826"} Jul 01 18:39:11.308296 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:11.307850 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" event={"ID":"bab11438-9d3e-4878-8984-47e11923a323","Type":"ContainerStarted","Data":"a5a423a9949507cebaadacfdb6dfc551d91574cec7fd31ce3439e77849818168"} Jul 01 18:39:11.339620 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:11.339573 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-f6xlc" podStartSLOduration=1.969679086 podStartE2EDuration="4.339557744s" totalImagesPullingTime="2.369878658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:11.338202879 +0000 UTC m=+109.941362794" watchObservedRunningTime="2026-07-01 18:39:11.339557744 +0000 UTC m=+109.942717637" Jul 01 18:39:13.531295 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.531267 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6wqvl"] Jul 01 18:39:13.533421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.533403 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.535984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.535953 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 18:39:13.535984 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.535964 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 18:39:13.536160 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.536010 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 18:39:13.536160 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.536072 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mtwh7\"" Jul 01 18:39:13.626696 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626662 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-textfile\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626696 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626701 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-wtmp\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-metrics-client-ca\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626775 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626803 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-sys\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626833 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vlh\" (UniqueName: \"kubernetes.io/projected/7715c2a2-d1e2-4110-b586-93c88f863638-kube-api-access-q8vlh\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.626853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-root\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.627016 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626881 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.627016 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.626948 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-tls\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.727900 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.727868 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-metrics-client-ca\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.727907 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.727936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-sys\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.727981 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vlh\" (UniqueName: \"kubernetes.io/projected/7715c2a2-d1e2-4110-b586-93c88f863638-kube-api-access-q8vlh\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728005 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-root\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728033 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728059 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-sys\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728093 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-root\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728109 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-tls\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728163 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-textfile\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-wtmp\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728553 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728370 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-wtmp\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728603 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728554 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-textfile\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728603 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728577 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-metrics-client-ca\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.728691 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.728678 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-accelerators-collector-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.730294 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.730266 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.730495 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.730476 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7715c2a2-d1e2-4110-b586-93c88f863638-node-exporter-tls\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.740348 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.740327 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vlh\" (UniqueName: \"kubernetes.io/projected/7715c2a2-d1e2-4110-b586-93c88f863638-kube-api-access-q8vlh\") pod \"node-exporter-6wqvl\" (UID: \"7715c2a2-d1e2-4110-b586-93c88f863638\") " pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.842537 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:13.842481 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6wqvl" Jul 01 18:39:13.852013 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:13.851987 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7715c2a2_d1e2_4110_b586_93c88f863638.slice/crio-929c6d945517556f26f92017bcfe06171f227603e2fc84d3d05dfa030b995c76 WatchSource:0}: Error finding container 929c6d945517556f26f92017bcfe06171f227603e2fc84d3d05dfa030b995c76: Status 404 returned error can't find the container with id 929c6d945517556f26f92017bcfe06171f227603e2fc84d3d05dfa030b995c76 Jul 01 18:39:14.320597 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:14.320558 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wqvl" event={"ID":"7715c2a2-d1e2-4110-b586-93c88f863638","Type":"ContainerStarted","Data":"929c6d945517556f26f92017bcfe06171f227603e2fc84d3d05dfa030b995c76"} Jul 01 18:39:15.325564 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:15.325518 2554 generic.go:356] "Generic (PLEG): container finished" podID="7715c2a2-d1e2-4110-b586-93c88f863638" containerID="894b5b8d8f8c17cb469e7fbc021cd58acbcdcb75a0f4528af88b841a66b159ed" exitCode=0 Jul 01 18:39:15.325917 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:15.325597 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wqvl" event={"ID":"7715c2a2-d1e2-4110-b586-93c88f863638","Type":"ContainerDied","Data":"894b5b8d8f8c17cb469e7fbc021cd58acbcdcb75a0f4528af88b841a66b159ed"} Jul 01 18:39:16.331418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.331380 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wqvl" event={"ID":"7715c2a2-d1e2-4110-b586-93c88f863638","Type":"ContainerStarted","Data":"e7c6edf2b7fe70252c2be14c31743706d7a2ff89d9a056cc4fc1413044b440dd"} Jul 01 18:39:16.331798 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.331504 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6wqvl" event={"ID":"7715c2a2-d1e2-4110-b586-93c88f863638","Type":"ContainerStarted","Data":"4f2a990bc57d33221267688e2881d1a0f0771d9d0a404e35aea3e737c3f116e5"} Jul 01 18:39:16.361951 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.361906 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6wqvl" podStartSLOduration=2.534632813 podStartE2EDuration="3.361891381s" totalImagesPullingTime="760.330511ms" totalInitContainerRuntime="66.928057ms" isStatefulPod=true podCreationTimestamp="2026-07-01 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:16.360433559 +0000 UTC m=+114.963593453" watchObservedRunningTime="2026-07-01 18:39:16.361891381 +0000 UTC m=+114.965051275" Jul 01 18:39:16.753521 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.753491 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7fcd97b69-86s5d"] Jul 01 18:39:16.756009 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.755995 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.758561 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.758541 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2pa0sru0s7m5h\"" Jul 01 18:39:16.758860 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.758828 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-n2t8q\"" Jul 01 18:39:16.758860 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.758844 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 18:39:16.759089 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.758827 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 18:39:16.759231 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.759215 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 18:39:16.759294 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.759248 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 18:39:16.759337 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.759311 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 18:39:16.778614 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.778595 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fcd97b69-86s5d"] Jul 01 18:39:16.859623 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859595 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-grpc-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859731 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859631 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859731 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859707 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859732 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1c3e5b9-6cad-4841-8582-b984e8f25847-metrics-client-ca\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859771 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859808 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859790 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfjjc\" (UniqueName: \"kubernetes.io/projected/f1c3e5b9-6cad-4841-8582-b984e8f25847-kube-api-access-jfjjc\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859944 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.859944 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.859826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960507 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960473 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960507 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960508 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfjjc\" (UniqueName: \"kubernetes.io/projected/f1c3e5b9-6cad-4841-8582-b984e8f25847-kube-api-access-jfjjc\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960529 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960557 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960584 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-grpc-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960604 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960819 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.960819 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.960689 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1c3e5b9-6cad-4841-8582-b984e8f25847-metrics-client-ca\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.961431 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.961404 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1c3e5b9-6cad-4841-8582-b984e8f25847-metrics-client-ca\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.963903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.963265 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.963903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.963314 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.963903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.963576 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-grpc-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.963903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.963779 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.963903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.963795 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.964199 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.964179 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f1c3e5b9-6cad-4841-8582-b984e8f25847-secret-thanos-querier-tls\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:16.987167 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:16.987139 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfjjc\" (UniqueName: \"kubernetes.io/projected/f1c3e5b9-6cad-4841-8582-b984e8f25847-kube-api-access-jfjjc\") pod \"thanos-querier-7fcd97b69-86s5d\" (UID: \"f1c3e5b9-6cad-4841-8582-b984e8f25847\") " pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:17.065327 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:17.065261 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:17.211478 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:17.211441 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fcd97b69-86s5d"] Jul 01 18:39:17.213911 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:17.213886 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c3e5b9_6cad_4841_8582_b984e8f25847.slice/crio-fc73dbbc7e8da16f9e3826d39f41990c6be73c949bc195ccffc1d0f47f3b5900 WatchSource:0}: Error finding container fc73dbbc7e8da16f9e3826d39f41990c6be73c949bc195ccffc1d0f47f3b5900: Status 404 returned error can't find the container with id fc73dbbc7e8da16f9e3826d39f41990c6be73c949bc195ccffc1d0f47f3b5900 Jul 01 18:39:17.216878 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:17.216860 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d65c54b89-nfd6t" Jul 01 18:39:17.338777 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:17.338703 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"fc73dbbc7e8da16f9e3826d39f41990c6be73c949bc195ccffc1d0f47f3b5900"} Jul 01 18:39:18.138087 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.138057 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn"] Jul 01 18:39:18.140143 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.140127 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.143552 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.143530 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bpzjx\"" Jul 01 18:39:18.143552 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.143546 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 01 18:39:18.159131 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.159112 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn"] Jul 01 18:39:18.161800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.160644 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75d5fd74f6-wjtks"] Jul 01 18:39:18.162951 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.162935 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.167054 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167036 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d1j0b0cm4ego3\"" Jul 01 18:39:18.167153 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167138 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-w4hs6\"" Jul 01 18:39:18.167212 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167165 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 18:39:18.167268 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167242 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 01 18:39:18.167317 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167272 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 01 18:39:18.167699 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.167684 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 01 18:39:18.169678 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.169655 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.169756 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.169733 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s9x6\" (UniqueName: \"kubernetes.io/projected/cea34acc-285a-4e38-9437-9b44f884ab4b-kube-api-access-4s9x6\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.184516 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.184496 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75d5fd74f6-wjtks"] Jul 01 18:39:18.270386 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270347 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-metrics-server-audit-profiles\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270549 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270398 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-client-certs\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270549 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270489 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.270549 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-tls\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270687 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:39:18.270567 2554 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 01 18:39:18.270687 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270613 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s9x6\" (UniqueName: \"kubernetes.io/projected/cea34acc-285a-4e38-9437-9b44f884ab4b-kube-api-access-4s9x6\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.270687 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:39:18.270631 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert podName:cea34acc-285a-4e38-9437-9b44f884ab4b nodeName:}" failed. No retries permitted until 2026-07-01 18:39:18.770611824 +0000 UTC m=+117.373771701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert") pod "monitoring-plugin-cd8656b8-hb9zn" (UID: "cea34acc-285a-4e38-9437-9b44f884ab4b") : secret "monitoring-plugin-cert" not found Jul 01 18:39:18.270687 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270682 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-client-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270843 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270720 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pt8\" (UniqueName: \"kubernetes.io/projected/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-kube-api-access-j6pt8\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270843 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270757 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.270917 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.270836 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-audit-log\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.287925 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.287896 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s9x6\" (UniqueName: \"kubernetes.io/projected/cea34acc-285a-4e38-9437-9b44f884ab4b-kube-api-access-4s9x6\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.372078 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372043 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-tls\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372135 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-client-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372169 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pt8\" (UniqueName: \"kubernetes.io/projected/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-kube-api-access-j6pt8\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372203 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372236 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-audit-log\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372294 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-metrics-server-audit-profiles\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372596 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-client-certs\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.372928 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.372861 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-audit-log\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.373030 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.373003 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.373561 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.373532 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-metrics-server-audit-profiles\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.375043 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.374984 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-client-certs\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.375178 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.375153 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-client-ca-bundle\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.375279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.375183 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-secret-metrics-server-tls\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.386625 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.386594 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pt8\" (UniqueName: \"kubernetes.io/projected/1499c9fb-badd-41de-ae9d-1a1df46fd6b6-kube-api-access-j6pt8\") pod \"metrics-server-75d5fd74f6-wjtks\" (UID: \"1499c9fb-badd-41de-ae9d-1a1df46fd6b6\") " pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.471360 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.471291 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:18.621123 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.621094 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75d5fd74f6-wjtks"] Jul 01 18:39:18.622514 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:18.622484 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1499c9fb_badd_41de_ae9d_1a1df46fd6b6.slice/crio-5ffa80d4de98ae5cb3688172458eadaa26cb1ba8f6ed1742ea76af58989a1f7d WatchSource:0}: Error finding container 5ffa80d4de98ae5cb3688172458eadaa26cb1ba8f6ed1742ea76af58989a1f7d: Status 404 returned error can't find the container with id 5ffa80d4de98ae5cb3688172458eadaa26cb1ba8f6ed1742ea76af58989a1f7d Jul 01 18:39:18.776236 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.776198 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.778837 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.778815 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cea34acc-285a-4e38-9437-9b44f884ab4b-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-hb9zn\" (UID: \"cea34acc-285a-4e38-9437-9b44f884ab4b\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:18.885903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.885869 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-695b6c96f5-sgjmq"] Jul 01 18:39:18.888975 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.888951 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.893896 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.893873 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 01 18:39:18.894049 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.894035 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 01 18:39:18.894161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.894134 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 01 18:39:18.895613 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.895590 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 01 18:39:18.896157 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.896109 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 01 18:39:18.896157 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.896125 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ww4fz\"" Jul 01 18:39:18.901978 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.901958 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 01 18:39:18.911375 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.911354 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-695b6c96f5-sgjmq"] Jul 01 18:39:18.978009 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.977980 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978018 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978050 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978133 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-serving-certs-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978285 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978167 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7xj2\" (UniqueName: \"kubernetes.io/projected/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-kube-api-access-j7xj2\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978285 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978198 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-federate-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978285 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978237 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-metrics-client-ca\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:18.978417 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:18.978350 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.049994 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.049915 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:19.079591 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079558 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079598 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079630 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079659 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-serving-certs-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079680 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7xj2\" (UniqueName: \"kubernetes.io/projected/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-kube-api-access-j7xj2\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079700 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-federate-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.079725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079719 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-metrics-client-ca\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.080008 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.079780 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.080697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.080671 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-serving-certs-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.081040 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.081011 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-metrics-client-ca\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.081199 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.081150 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-trusted-ca-bundle\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.082513 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.082493 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-telemeter-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.082582 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.082548 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.082696 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.082678 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-federate-client-tls\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.082982 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.082960 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-secret-telemeter-client\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.092675 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.092653 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7xj2\" (UniqueName: \"kubernetes.io/projected/54b9a7ca-e2de-444a-a4cf-606301dcb6ec-kube-api-access-j7xj2\") pod \"telemeter-client-695b6c96f5-sgjmq\" (UID: \"54b9a7ca-e2de-444a-a4cf-606301dcb6ec\") " pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.201740 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.201706 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" Jul 01 18:39:19.353619 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.353519 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"06805fdf98060a7d11bb98e6f147f4cc2a6a1f26a9c06544d3db644a0243a3c2"} Jul 01 18:39:19.355148 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.355080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" event={"ID":"1499c9fb-badd-41de-ae9d-1a1df46fd6b6","Type":"ContainerStarted","Data":"5ffa80d4de98ae5cb3688172458eadaa26cb1ba8f6ed1742ea76af58989a1f7d"} Jul 01 18:39:19.401544 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.401501 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-695b6c96f5-sgjmq"] Jul 01 18:39:19.403953 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:19.403874 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b9a7ca_e2de_444a_a4cf_606301dcb6ec.slice/crio-d7ea0c3bf5d791934fef5a7163765fdc27f353b931db88e2d7dec2d1cfa839b5 WatchSource:0}: Error finding container d7ea0c3bf5d791934fef5a7163765fdc27f353b931db88e2d7dec2d1cfa839b5: Status 404 returned error can't find the container with id d7ea0c3bf5d791934fef5a7163765fdc27f353b931db88e2d7dec2d1cfa839b5 Jul 01 18:39:19.419164 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:19.419141 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn"] Jul 01 18:39:19.422378 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:19.422273 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea34acc_285a_4e38_9437_9b44f884ab4b.slice/crio-f80088134d4e08c36452db70bc1b8819558f081d5f802aaa1f7e71cd589b4129 WatchSource:0}: Error finding container f80088134d4e08c36452db70bc1b8819558f081d5f802aaa1f7e71cd589b4129: Status 404 returned error can't find the container with id f80088134d4e08c36452db70bc1b8819558f081d5f802aaa1f7e71cd589b4129 Jul 01 18:39:20.312286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.310859 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zr9jd" Jul 01 18:39:20.361571 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.361527 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" event={"ID":"54b9a7ca-e2de-444a-a4cf-606301dcb6ec","Type":"ContainerStarted","Data":"d7ea0c3bf5d791934fef5a7163765fdc27f353b931db88e2d7dec2d1cfa839b5"} Jul 01 18:39:20.363166 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.363102 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" event={"ID":"cea34acc-285a-4e38-9437-9b44f884ab4b","Type":"ContainerStarted","Data":"f80088134d4e08c36452db70bc1b8819558f081d5f802aaa1f7e71cd589b4129"} Jul 01 18:39:20.366913 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.366790 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"c0a5a9ada9d3b684d7903514456cd9d3b3eed5d118d01ae99decbd8331197ecf"} Jul 01 18:39:20.366913 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.366862 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"a20bceb315d11460b7110180b93ba644df3ee2d28e6dbf9b59c674cca6ee907a"} Jul 01 18:39:20.421425 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:20.421373 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" podUID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" containerName="registry" containerID="cri-o://dd1864423d472348d69f71d64acf48b62a806deaa040013f0483266baedd5481" gracePeriod=30 Jul 01 18:39:21.373004 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.372967 2554 generic.go:356] "Generic (PLEG): container finished" podID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" containerID="dd1864423d472348d69f71d64acf48b62a806deaa040013f0483266baedd5481" exitCode=0 Jul 01 18:39:21.373155 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.373042 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" event={"ID":"dc023fe8-4e56-477d-99a1-9fdbe29174d0","Type":"ContainerDied","Data":"dd1864423d472348d69f71d64acf48b62a806deaa040013f0483266baedd5481"} Jul 01 18:39:21.598494 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.598472 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:39:21.711236 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711215 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711338 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711253 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711338 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711279 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711338 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711297 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711338 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711331 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711374 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cjrt\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711397 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.711684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.711423 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted\") pod \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\" (UID: \"dc023fe8-4e56-477d-99a1-9fdbe29174d0\") " Jul 01 18:39:21.712222 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.712192 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:21.712669 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.712602 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:39:21.714186 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.714135 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:21.714417 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.714379 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:21.714775 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.714743 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:39:21.719094 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.718507 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt" (OuterVolumeSpecName: "kube-api-access-4cjrt") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "kube-api-access-4cjrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:21.719094 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.718939 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:39:21.724753 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.724726 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dc023fe8-4e56-477d-99a1-9fdbe29174d0" (UID: "dc023fe8-4e56-477d-99a1-9fdbe29174d0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 18:39:21.812733 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812703 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-bound-sa-token\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812737 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-image-registry-private-configuration\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812752 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dc023fe8-4e56-477d-99a1-9fdbe29174d0-installation-pull-secrets\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812765 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-trusted-ca\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812780 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-certificates\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812792 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cjrt\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-kube-api-access-4cjrt\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812805 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dc023fe8-4e56-477d-99a1-9fdbe29174d0-registry-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:21.812856 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:21.812817 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dc023fe8-4e56-477d-99a1-9fdbe29174d0-ca-trust-extracted\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:39:22.378858 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.378817 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" event={"ID":"54b9a7ca-e2de-444a-a4cf-606301dcb6ec","Type":"ContainerStarted","Data":"88a421a92164a84824fcc0ed1403ba0bee9a9708eee0c2806670bd6a8aa893e1"} Jul 01 18:39:22.380370 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.380340 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" event={"ID":"cea34acc-285a-4e38-9437-9b44f884ab4b","Type":"ContainerStarted","Data":"aaec317921bb5a3663b8f394924d59470935e4b5051dc74a5046e3b42211d526"} Jul 01 18:39:22.380520 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.380490 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:22.384085 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.384057 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"896e2d6ed5bd3bd5533babaa453f54e681dc39f69d50f1d990ed591f337da9e6"} Jul 01 18:39:22.384256 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.384135 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:22.384256 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.384181 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"1349967ab8192cbccded7c5ae5c2684423cf91a0fdb9b935fe38f127849bc783"} Jul 01 18:39:22.384256 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.384224 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" event={"ID":"f1c3e5b9-6cad-4841-8582-b984e8f25847","Type":"ContainerStarted","Data":"5490645fc4919acba07829ca1618c63197b0b4347a99aaeefbf9d967cf766987"} Jul 01 18:39:22.385504 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.385477 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" event={"ID":"dc023fe8-4e56-477d-99a1-9fdbe29174d0","Type":"ContainerDied","Data":"1ae40a6726d4923e40c44db764621ecf853b779f1d672eb47da70cab0f9bb8d0"} Jul 01 18:39:22.385504 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.385498 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bc666f464-qmqsp" Jul 01 18:39:22.385697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.385539 2554 scope.go:117] "RemoveContainer" containerID="dd1864423d472348d69f71d64acf48b62a806deaa040013f0483266baedd5481" Jul 01 18:39:22.387815 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.387581 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" Jul 01 18:39:22.388100 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.388016 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" event={"ID":"1499c9fb-badd-41de-ae9d-1a1df46fd6b6","Type":"ContainerStarted","Data":"7a79d46757bce3c2fe687573d7200ba60f53fffb16553128063cfc3fbabfb09a"} Jul 01 18:39:22.400799 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.400751 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-hb9zn" podStartSLOduration=2.208918854 podStartE2EDuration="4.400733271s" totalImagesPullingTime="2.191814417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:22.399940234 +0000 UTC m=+121.003100149" watchObservedRunningTime="2026-07-01 18:39:22.400733271 +0000 UTC m=+121.003893165" Jul 01 18:39:22.428911 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.428849 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" podStartSLOduration=1.43761911 podStartE2EDuration="4.428827344s" totalImagesPullingTime="2.991208234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:22.427293713 +0000 UTC m=+121.030453608" watchObservedRunningTime="2026-07-01 18:39:22.428827344 +0000 UTC m=+121.031987239" Jul 01 18:39:22.447660 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.446239 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:39:22.454864 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.454836 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bc666f464-qmqsp"] Jul 01 18:39:22.479765 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:22.479713 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" podStartSLOduration=2.29918966 podStartE2EDuration="6.479691301s" totalImagesPullingTime="4.180501641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:22.478588844 +0000 UTC m=+121.081748749" watchObservedRunningTime="2026-07-01 18:39:22.479691301 +0000 UTC m=+121.082851198" Jul 01 18:39:23.393904 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:23.393866 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" event={"ID":"54b9a7ca-e2de-444a-a4cf-606301dcb6ec","Type":"ContainerStarted","Data":"d137a7f4e48437cdf16b1499a90088375af2cd8b834a8f66d39c28dc3830b744"} Jul 01 18:39:23.394351 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:23.393963 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" event={"ID":"54b9a7ca-e2de-444a-a4cf-606301dcb6ec","Type":"ContainerStarted","Data":"c84f3a161131ddde00a06d3b1ff682c101c9bddbf94138863030c7e0a1cbf545"} Jul 01 18:39:23.421783 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:23.421729 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-695b6c96f5-sgjmq" podStartSLOduration=2.044476362 podStartE2EDuration="5.421711908s" totalImagesPullingTime="3.377235546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:23.420220685 +0000 UTC m=+122.023380613" watchObservedRunningTime="2026-07-01 18:39:23.421711908 +0000 UTC m=+122.024871802" Jul 01 18:39:23.823602 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:23.823573 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" path="/var/lib/kubelet/pods/dc023fe8-4e56-477d-99a1-9fdbe29174d0/volumes" Jul 01 18:39:28.163435 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.163405 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:39:28.163814 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.163747 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" containerName="registry" Jul 01 18:39:28.163814 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.163760 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" containerName="registry" Jul 01 18:39:28.163814 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.163806 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc023fe8-4e56-477d-99a1-9fdbe29174d0" containerName="registry" Jul 01 18:39:28.167599 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.167582 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.177486 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.177446 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 18:39:28.180030 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.180011 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:39:28.267164 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267287 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267180 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267341 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267289 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267341 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267329 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267375 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267394 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.267546 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.267412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75sqg\" (UniqueName: \"kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.367940 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.367901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.367940 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.367938 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368135 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.367979 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368135 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368008 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368135 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368034 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75sqg\" (UniqueName: \"kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368135 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368072 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368135 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368127 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.368899 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368873 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.369019 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.368984 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.369079 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.369020 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.369079 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.369059 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.370573 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.370549 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.370683 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.370659 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.380800 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.380778 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75sqg\" (UniqueName: \"kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg\") pod \"console-7c4958d649-kw8fj\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.401067 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.401050 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7fcd97b69-86s5d" Jul 01 18:39:28.476739 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.476673 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:28.610715 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:28.610625 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:39:28.612794 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:28.612768 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4011118a_1ad3_491c_9439_1cc20b405f7a.slice/crio-d518dcf90797a8006ed999dbef2934523aca50818c50a3ac79df3b7317adeb2f WatchSource:0}: Error finding container d518dcf90797a8006ed999dbef2934523aca50818c50a3ac79df3b7317adeb2f: Status 404 returned error can't find the container with id d518dcf90797a8006ed999dbef2934523aca50818c50a3ac79df3b7317adeb2f Jul 01 18:39:29.419179 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:29.419137 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4958d649-kw8fj" event={"ID":"4011118a-1ad3-491c-9439-1cc20b405f7a","Type":"ContainerStarted","Data":"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119"} Jul 01 18:39:29.419545 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:29.419197 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4958d649-kw8fj" event={"ID":"4011118a-1ad3-491c-9439-1cc20b405f7a","Type":"ContainerStarted","Data":"d518dcf90797a8006ed999dbef2934523aca50818c50a3ac79df3b7317adeb2f"} Jul 01 18:39:29.442432 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:29.442382 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c4958d649-kw8fj" podStartSLOduration=1.442367405 podStartE2EDuration="1.442367405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:29.440161059 +0000 UTC m=+128.043320953" watchObservedRunningTime="2026-07-01 18:39:29.442367405 +0000 UTC m=+128.045527368" Jul 01 18:39:30.586078 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:30.586043 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:39:30.588352 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:30.588329 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c9cac5a-22da-46f2-a514-049a7075ea4a-metrics-certs\") pod \"network-metrics-daemon-6hvdk\" (UID: \"2c9cac5a-22da-46f2-a514-049a7075ea4a\") " pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:39:30.644437 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:30.644415 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vz9\"" Jul 01 18:39:30.652127 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:30.652104 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6hvdk" Jul 01 18:39:30.796057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:30.796036 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6hvdk"] Jul 01 18:39:30.798349 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:39:30.798320 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9cac5a_22da_46f2_a514_049a7075ea4a.slice/crio-2c36412079f69955dba99ff51f19a444aeb9ec9ecd925cb567ebb45486eea3c5 WatchSource:0}: Error finding container 2c36412079f69955dba99ff51f19a444aeb9ec9ecd925cb567ebb45486eea3c5: Status 404 returned error can't find the container with id 2c36412079f69955dba99ff51f19a444aeb9ec9ecd925cb567ebb45486eea3c5 Jul 01 18:39:31.432346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:31.432311 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6hvdk" event={"ID":"2c9cac5a-22da-46f2-a514-049a7075ea4a","Type":"ContainerStarted","Data":"2c36412079f69955dba99ff51f19a444aeb9ec9ecd925cb567ebb45486eea3c5"} Jul 01 18:39:32.438008 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:32.437973 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6hvdk" event={"ID":"2c9cac5a-22da-46f2-a514-049a7075ea4a","Type":"ContainerStarted","Data":"e31d6f1142253a31a5646f74c7f5eeffa9f8c1337710e36e3e2dd446f8020809"} Jul 01 18:39:32.438467 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:32.438025 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6hvdk" event={"ID":"2c9cac5a-22da-46f2-a514-049a7075ea4a","Type":"ContainerStarted","Data":"112afc5f08b937df9d106baee4449287fc59fae6ab9e9c32b70e8ad474f9bb2e"} Jul 01 18:39:32.461302 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:32.461256 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6hvdk" podStartSLOduration=129.599196549 podStartE2EDuration="2m10.46124105s" totalImagesPullingTime="862.044501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:37:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:39:32.45998757 +0000 UTC m=+131.063147464" watchObservedRunningTime="2026-07-01 18:39:32.46124105 +0000 UTC m=+131.064400944" Jul 01 18:39:37.954988 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:37.954955 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b72ng_e455c3a6-5452-4971-8b72-072592a96d92/serve-healthcheck-canary/0.log" Jul 01 18:39:38.472374 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:38.472349 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:38.472562 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:38.472398 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:38.476806 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:38.476787 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:38.476903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:38.476833 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:38.482158 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:38.482139 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:39.468109 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:39.468087 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:39:39.544801 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:39.544772 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:39:49.507853 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:49.507820 2554 generic.go:356] "Generic (PLEG): container finished" podID="258d12c6-0758-4352-9dfc-b55aa56157e4" containerID="60a86e882c8739bbb95b07dcf7db30a0fc74c3b34845cfe5f2514f0a7a38088f" exitCode=0 Jul 01 18:39:49.508312 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:49.507891 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" event={"ID":"258d12c6-0758-4352-9dfc-b55aa56157e4","Type":"ContainerDied","Data":"60a86e882c8739bbb95b07dcf7db30a0fc74c3b34845cfe5f2514f0a7a38088f"} Jul 01 18:39:49.508312 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:49.508209 2554 scope.go:117] "RemoveContainer" containerID="60a86e882c8739bbb95b07dcf7db30a0fc74c3b34845cfe5f2514f0a7a38088f" Jul 01 18:39:50.513567 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:50.513443 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-wzlmn" event={"ID":"258d12c6-0758-4352-9dfc-b55aa56157e4","Type":"ContainerStarted","Data":"4d40d3d84b47ad1b1a14dc5b59290e10ffdc40d61b315e810a0ed6763bae38f5"} Jul 01 18:39:54.538269 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:54.538234 2554 generic.go:356] "Generic (PLEG): container finished" podID="87f2696b-218e-4902-8a03-71a1c19b90c7" containerID="b410e40f106d3fa373c40c1a4ef8450e1f4a81b28c2126b282a775662ece873b" exitCode=0 Jul 01 18:39:54.538657 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:54.538276 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" event={"ID":"87f2696b-218e-4902-8a03-71a1c19b90c7","Type":"ContainerDied","Data":"b410e40f106d3fa373c40c1a4ef8450e1f4a81b28c2126b282a775662ece873b"} Jul 01 18:39:54.538657 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:54.538600 2554 scope.go:117] "RemoveContainer" containerID="b410e40f106d3fa373c40c1a4ef8450e1f4a81b28c2126b282a775662ece873b" Jul 01 18:39:55.544370 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:55.544338 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-m55vr" event={"ID":"87f2696b-218e-4902-8a03-71a1c19b90c7","Type":"ContainerStarted","Data":"33affbabe52a35a7800f04c315fc77cae3d678e6a2d07f8740f73c89776d0cb3"} Jul 01 18:39:58.476668 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:58.476639 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:39:58.480983 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:39:58.480958 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75d5fd74f6-wjtks" Jul 01 18:40:04.564905 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.564844 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85cdf66d46-m5dgv" podUID="b7ea25d9-886e-4346-a42a-828fc4944ad4" containerName="console" containerID="cri-o://13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721" gracePeriod=15 Jul 01 18:40:04.810937 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.810908 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:40:04.875556 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875488 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rbsd\" (UniqueName: \"kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.875556 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875522 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.875556 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875548 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.875755 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875584 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.875755 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875614 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.875755 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875698 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert\") pod \"b7ea25d9-886e-4346-a42a-828fc4944ad4\" (UID: \"b7ea25d9-886e-4346-a42a-828fc4944ad4\") " Jul 01 18:40:04.876021 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.875994 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca" (OuterVolumeSpecName: "service-ca") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:04.876114 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.876008 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:04.876162 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.876108 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config" (OuterVolumeSpecName: "console-config") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:40:04.877729 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.877699 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:04.877814 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.877757 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:40:04.877898 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.877837 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd" (OuterVolumeSpecName: "kube-api-access-5rbsd") pod "b7ea25d9-886e-4346-a42a-828fc4944ad4" (UID: "b7ea25d9-886e-4346-a42a-828fc4944ad4"). InnerVolumeSpecName "kube-api-access-5rbsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:40:04.976292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976270 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:04.976292 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976291 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rbsd\" (UniqueName: \"kubernetes.io/projected/b7ea25d9-886e-4346-a42a-828fc4944ad4-kube-api-access-5rbsd\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:04.976419 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976302 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-service-ca\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:04.976419 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976312 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-oauth-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:04.976419 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976320 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-oauth-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:04.976419 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:04.976328 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b7ea25d9-886e-4346-a42a-828fc4944ad4-console-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:40:05.581781 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.581750 2554 generic.go:356] "Generic (PLEG): container finished" podID="b7ea25d9-886e-4346-a42a-828fc4944ad4" containerID="13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721" exitCode=0 Jul 01 18:40:05.582183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.581790 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdf66d46-m5dgv" Jul 01 18:40:05.582183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.581839 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdf66d46-m5dgv" event={"ID":"b7ea25d9-886e-4346-a42a-828fc4944ad4","Type":"ContainerDied","Data":"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721"} Jul 01 18:40:05.582183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.581884 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdf66d46-m5dgv" event={"ID":"b7ea25d9-886e-4346-a42a-828fc4944ad4","Type":"ContainerDied","Data":"2d30cb141006a848792249d9059f7d0c37d7b6bff23f6d8117f7d1565ffa15c7"} Jul 01 18:40:05.582183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.581903 2554 scope.go:117] "RemoveContainer" containerID="13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721" Jul 01 18:40:05.590914 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.590894 2554 scope.go:117] "RemoveContainer" containerID="13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721" Jul 01 18:40:05.591187 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:40:05.591166 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721\": container with ID starting with 13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721 not found: ID does not exist" containerID="13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721" Jul 01 18:40:05.591244 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.591195 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721"} err="failed to get container status \"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721\": rpc error: code = NotFound desc = could not find container \"13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721\": container with ID starting with 13818facd6f607dde24a1db6ea7b853c67fe75e53e86c89a3ba297e14b6a9721 not found: ID does not exist" Jul 01 18:40:05.604718 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.604698 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:40:05.610672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.610653 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85cdf66d46-m5dgv"] Jul 01 18:40:05.824286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:05.824256 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7ea25d9-886e-4346-a42a-828fc4944ad4" path="/var/lib/kubelet/pods/b7ea25d9-886e-4346-a42a-828fc4944ad4/volumes" Jul 01 18:40:33.878003 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.877970 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 18:40:33.878408 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.878243 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7ea25d9-886e-4346-a42a-828fc4944ad4" containerName="console" Jul 01 18:40:33.878408 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.878254 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7ea25d9-886e-4346-a42a-828fc4944ad4" containerName="console" Jul 01 18:40:33.878408 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.878330 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7ea25d9-886e-4346-a42a-828fc4944ad4" containerName="console" Jul 01 18:40:33.881365 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.881347 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:33.919636 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:33.919611 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 18:40:34.015831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.015792 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.015831 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.015834 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.016057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.015854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.016057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.015943 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.016057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.015978 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.016057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.016002 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lh8s\" (UniqueName: \"kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.016226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.016068 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.116902 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.116873 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.116907 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.116927 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.116968 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.116984 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117026 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117001 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lh8s\" (UniqueName: \"kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117191 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117165 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117774 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117748 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117912 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117852 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.117912 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117899 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.118040 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.117899 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.119506 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.119486 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.119594 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.119518 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.125776 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.125754 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lh8s\" (UniqueName: \"kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s\") pod \"console-549b475fd4-hj6tb\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.190280 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.190218 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:34.330071 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.330038 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 18:40:34.332890 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:40:34.332862 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895c7176_10e1_4112_bde2_63ba4542031f.slice/crio-c297e12b4079686d8477651d543229166e90b6a0ef620c4c37095c58db2b9480 WatchSource:0}: Error finding container c297e12b4079686d8477651d543229166e90b6a0ef620c4c37095c58db2b9480: Status 404 returned error can't find the container with id c297e12b4079686d8477651d543229166e90b6a0ef620c4c37095c58db2b9480 Jul 01 18:40:34.692037 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.692008 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b475fd4-hj6tb" event={"ID":"895c7176-10e1-4112-bde2-63ba4542031f","Type":"ContainerStarted","Data":"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de"} Jul 01 18:40:34.692211 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.692056 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b475fd4-hj6tb" event={"ID":"895c7176-10e1-4112-bde2-63ba4542031f","Type":"ContainerStarted","Data":"c297e12b4079686d8477651d543229166e90b6a0ef620c4c37095c58db2b9480"} Jul 01 18:40:34.714525 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:34.714477 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-549b475fd4-hj6tb" podStartSLOduration=1.714437716 podStartE2EDuration="1.714437716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:40:34.713953628 +0000 UTC m=+193.317113524" watchObservedRunningTime="2026-07-01 18:40:34.714437716 +0000 UTC m=+193.317597610" Jul 01 18:40:44.190629 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:44.190589 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:44.191118 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:44.190668 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:44.195476 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:44.195436 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:44.736304 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:44.736274 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 18:40:44.801186 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:40:44.801152 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:41:09.821436 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:09.821397 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c4958d649-kw8fj" podUID="4011118a-1ad3-491c-9439-1cc20b405f7a" containerName="console" containerID="cri-o://8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119" gracePeriod=15 Jul 01 18:41:10.059471 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.059438 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:41:10.113987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.113908 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.113987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.113945 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75sqg\" (UniqueName: \"kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.113987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.113969 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.113987 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.113990 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.114257 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114099 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.114257 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114148 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.114358 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114254 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert\") pod \"4011118a-1ad3-491c-9439-1cc20b405f7a\" (UID: \"4011118a-1ad3-491c-9439-1cc20b405f7a\") " Jul 01 18:41:10.114662 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114335 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config" (OuterVolumeSpecName: "console-config") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:41:10.114662 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114561 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca" (OuterVolumeSpecName: "service-ca") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:41:10.114662 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114574 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:41:10.114863 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.114758 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:41:10.116257 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.116222 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:41:10.116619 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.116599 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:41:10.116691 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.116600 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg" (OuterVolumeSpecName: "kube-api-access-75sqg") pod "4011118a-1ad3-491c-9439-1cc20b405f7a" (UID: "4011118a-1ad3-491c-9439-1cc20b405f7a"). InnerVolumeSpecName "kube-api-access-75sqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 18:41:10.214904 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214860 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-trusted-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.214904 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214897 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.214904 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214907 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-oauth-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.214904 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214916 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-service-ca\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.215161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214925 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75sqg\" (UniqueName: \"kubernetes.io/projected/4011118a-1ad3-491c-9439-1cc20b405f7a-kube-api-access-75sqg\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.215161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214934 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4011118a-1ad3-491c-9439-1cc20b405f7a-console-oauth-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.215161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.214943 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4011118a-1ad3-491c-9439-1cc20b405f7a-console-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:41:10.832313 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.832280 2554 generic.go:356] "Generic (PLEG): container finished" podID="4011118a-1ad3-491c-9439-1cc20b405f7a" containerID="8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119" exitCode=0 Jul 01 18:41:10.832725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.832330 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c4958d649-kw8fj" Jul 01 18:41:10.832725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.832373 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4958d649-kw8fj" event={"ID":"4011118a-1ad3-491c-9439-1cc20b405f7a","Type":"ContainerDied","Data":"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119"} Jul 01 18:41:10.832725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.832425 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c4958d649-kw8fj" event={"ID":"4011118a-1ad3-491c-9439-1cc20b405f7a","Type":"ContainerDied","Data":"d518dcf90797a8006ed999dbef2934523aca50818c50a3ac79df3b7317adeb2f"} Jul 01 18:41:10.832725 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.832445 2554 scope.go:117] "RemoveContainer" containerID="8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119" Jul 01 18:41:10.841246 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.841230 2554 scope.go:117] "RemoveContainer" containerID="8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119" Jul 01 18:41:10.841514 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:41:10.841499 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119\": container with ID starting with 8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119 not found: ID does not exist" containerID="8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119" Jul 01 18:41:10.841565 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.841522 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119"} err="failed to get container status \"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119\": rpc error: code = NotFound desc = could not find container \"8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119\": container with ID starting with 8bbc2c481c5606d9714069344cb82678494a2e2245d8b62b6b9413bbd3e68119 not found: ID does not exist" Jul 01 18:41:10.856206 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.856181 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:41:10.861447 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:10.861425 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c4958d649-kw8fj"] Jul 01 18:41:11.824520 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:41:11.824487 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4011118a-1ad3-491c-9439-1cc20b405f7a" path="/var/lib/kubelet/pods/4011118a-1ad3-491c-9439-1cc20b405f7a/volumes" Jul 01 18:42:21.766826 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:21.766797 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:42:21.768931 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:21.768909 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:42:21.774247 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:21.774227 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 18:42:31.500105 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.500072 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg"] Jul 01 18:42:31.502286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.500362 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4011118a-1ad3-491c-9439-1cc20b405f7a" containerName="console" Jul 01 18:42:31.502286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.500373 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4011118a-1ad3-491c-9439-1cc20b405f7a" containerName="console" Jul 01 18:42:31.502286 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.500436 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4011118a-1ad3-491c-9439-1cc20b405f7a" containerName="console" Jul 01 18:42:31.503215 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.503201 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.506073 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.506049 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 01 18:42:31.506073 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.506063 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-hz2f2\"" Jul 01 18:42:31.506225 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.506054 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 01 18:42:31.506410 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.506389 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 01 18:42:31.522928 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.522906 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg"] Jul 01 18:42:31.591192 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.591159 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzwx5\" (UniqueName: \"kubernetes.io/projected/69290931-1fdf-4f03-9f59-898d7eeaf6e7-kube-api-access-wzwx5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.591324 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.591211 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/69290931-1fdf-4f03-9f59-898d7eeaf6e7-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.691694 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.691661 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzwx5\" (UniqueName: \"kubernetes.io/projected/69290931-1fdf-4f03-9f59-898d7eeaf6e7-kube-api-access-wzwx5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.691839 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.691710 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/69290931-1fdf-4f03-9f59-898d7eeaf6e7-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.694008 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.693987 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/69290931-1fdf-4f03-9f59-898d7eeaf6e7-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.708803 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.708777 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzwx5\" (UniqueName: \"kubernetes.io/projected/69290931-1fdf-4f03-9f59-898d7eeaf6e7-kube-api-access-wzwx5\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg\" (UID: \"69290931-1fdf-4f03-9f59-898d7eeaf6e7\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.814190 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.814106 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:31.950645 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.950619 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg"] Jul 01 18:42:31.953356 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:42:31.953320 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69290931_1fdf_4f03_9f59_898d7eeaf6e7.slice/crio-20c3ab8d52535735635c41f69ff06d3397bc1ba9b1fdece275bd30257e40742e WatchSource:0}: Error finding container 20c3ab8d52535735635c41f69ff06d3397bc1ba9b1fdece275bd30257e40742e: Status 404 returned error can't find the container with id 20c3ab8d52535735635c41f69ff06d3397bc1ba9b1fdece275bd30257e40742e Jul 01 18:42:31.955235 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:31.955217 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:42:32.147122 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:32.147045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" event={"ID":"69290931-1fdf-4f03-9f59-898d7eeaf6e7","Type":"ContainerStarted","Data":"20c3ab8d52535735635c41f69ff06d3397bc1ba9b1fdece275bd30257e40742e"} Jul 01 18:42:36.170753 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.170714 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" event={"ID":"69290931-1fdf-4f03-9f59-898d7eeaf6e7","Type":"ContainerStarted","Data":"5adc74656f943d21a41d523586b5c5afd68fe767a6c6a3dd0970fb7dc006ba4c"} Jul 01 18:42:36.171314 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.170843 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:42:36.194288 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.194189 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" podStartSLOduration=1.204667204 podStartE2EDuration="5.1941711s" totalImagesPullingTime="3.989503896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:42:36.191972861 +0000 UTC m=+314.795132761" watchObservedRunningTime="2026-07-01 18:42:36.1941711 +0000 UTC m=+314.797331042" Jul 01 18:42:36.747764 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.747729 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-9qxp2"] Jul 01 18:42:36.750946 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.750927 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:36.753400 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.753377 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 01 18:42:36.753529 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.753416 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 01 18:42:36.753529 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.753442 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-w4hbd\"" Jul 01 18:42:36.763672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.763649 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-9qxp2"] Jul 01 18:42:36.938056 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.938022 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c9a10991-d6c3-4d56-a5e5-b626f62133c4-cabundle0\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:36.938215 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.938123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-kube-api-access-cbgjb\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:36.938215 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:36.938171 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.039511 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.039397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-kube-api-access-cbgjb\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.039511 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.039493 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.039749 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.039522 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c9a10991-d6c3-4d56-a5e5-b626f62133c4-cabundle0\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.039749 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:42:37.039656 2554 secret.go:281] references non-existent secret key: ca.crt Jul 01 18:42:37.039749 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:42:37.039679 2554 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 01 18:42:37.039749 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:42:37.039691 2554 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-9qxp2: references non-existent secret key: ca.crt Jul 01 18:42:37.039968 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:42:37.039761 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates podName:c9a10991-d6c3-4d56-a5e5-b626f62133c4 nodeName:}" failed. No retries permitted until 2026-07-01 18:42:37.539737866 +0000 UTC m=+316.142897752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates") pod "keda-operator-68bff59c-9qxp2" (UID: "c9a10991-d6c3-4d56-a5e5-b626f62133c4") : references non-existent secret key: ca.crt Jul 01 18:42:37.040205 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.040185 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c9a10991-d6c3-4d56-a5e5-b626f62133c4-cabundle0\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.050859 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.050834 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbgjb\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-kube-api-access-cbgjb\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.181565 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.181528 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-znkvp"] Jul 01 18:42:37.184942 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.184919 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.187302 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.187276 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 01 18:42:37.195061 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.195031 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-znkvp"] Jul 01 18:42:37.341920 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.341833 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stsch\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-kube-api-access-stsch\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.342070 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.341959 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a3a58360-563f-4815-8274-ba2039a5ccc7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.342070 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.342034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-certificates\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.443511 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.443475 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a3a58360-563f-4815-8274-ba2039a5ccc7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.443684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.443569 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-certificates\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.443684 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.443626 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stsch\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-kube-api-access-stsch\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.443936 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.443914 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/a3a58360-563f-4815-8274-ba2039a5ccc7-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.446138 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.446113 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-certificates\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.453301 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.453279 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stsch\" (UniqueName: \"kubernetes.io/projected/a3a58360-563f-4815-8274-ba2039a5ccc7-kube-api-access-stsch\") pod \"keda-metrics-apiserver-559f75f947-znkvp\" (UID: \"a3a58360-563f-4815-8274-ba2039a5ccc7\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.496058 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.496024 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:37.544096 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.544053 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.546921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.546892 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c9a10991-d6c3-4d56-a5e5-b626f62133c4-certificates\") pod \"keda-operator-68bff59c-9qxp2\" (UID: \"c9a10991-d6c3-4d56-a5e5-b626f62133c4\") " pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.644790 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.644704 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-znkvp"] Jul 01 18:42:37.648376 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:42:37.648349 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a58360_563f_4815_8274_ba2039a5ccc7.slice/crio-e7e1b1e65e2acb9a9ee77b0865412a80cb7250ccc06745a9e3d98199cbaf7247 WatchSource:0}: Error finding container e7e1b1e65e2acb9a9ee77b0865412a80cb7250ccc06745a9e3d98199cbaf7247: Status 404 returned error can't find the container with id e7e1b1e65e2acb9a9ee77b0865412a80cb7250ccc06745a9e3d98199cbaf7247 Jul 01 18:42:37.661593 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.661564 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:37.817691 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:37.817657 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-9qxp2"] Jul 01 18:42:37.820786 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:42:37.820754 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a10991_d6c3_4d56_a5e5_b626f62133c4.slice/crio-2d14153cbe8ac987a8965df864dd44b9ec0836468666e83a0e144a813d84806c WatchSource:0}: Error finding container 2d14153cbe8ac987a8965df864dd44b9ec0836468666e83a0e144a813d84806c: Status 404 returned error can't find the container with id 2d14153cbe8ac987a8965df864dd44b9ec0836468666e83a0e144a813d84806c Jul 01 18:42:38.181320 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:38.181279 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" event={"ID":"a3a58360-563f-4815-8274-ba2039a5ccc7","Type":"ContainerStarted","Data":"e7e1b1e65e2acb9a9ee77b0865412a80cb7250ccc06745a9e3d98199cbaf7247"} Jul 01 18:42:38.182316 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:38.182293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-9qxp2" event={"ID":"c9a10991-d6c3-4d56-a5e5-b626f62133c4","Type":"ContainerStarted","Data":"2d14153cbe8ac987a8965df864dd44b9ec0836468666e83a0e144a813d84806c"} Jul 01 18:42:43.208313 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.208274 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" event={"ID":"a3a58360-563f-4815-8274-ba2039a5ccc7","Type":"ContainerStarted","Data":"cc5dbe6672954fea04e77028db3b0a811fc9d3b0441220c5f6ee95c69edae7ca"} Jul 01 18:42:43.208806 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.208438 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:43.209721 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.209694 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-9qxp2" event={"ID":"c9a10991-d6c3-4d56-a5e5-b626f62133c4","Type":"ContainerStarted","Data":"f4bf312c6a6af77086d3b270835ea2dcd4b6994d2094a6287dc2f2e35d0c3b16"} Jul 01 18:42:43.209861 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.209844 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:42:43.227387 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.227337 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" podStartSLOduration=0.853970236 podStartE2EDuration="6.227316189s" totalImagesPullingTime="5.373345953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:42:43.226055087 +0000 UTC m=+321.829214981" watchObservedRunningTime="2026-07-01 18:42:43.227316189 +0000 UTC m=+321.830476080" Jul 01 18:42:43.244957 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:43.244907 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-9qxp2" podStartSLOduration=2.038132076 podStartE2EDuration="7.244889935s" totalImagesPullingTime="5.206757859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:42:43.24394122 +0000 UTC m=+321.847101115" watchObservedRunningTime="2026-07-01 18:42:43.244889935 +0000 UTC m=+321.848049828" Jul 01 18:42:54.219714 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:54.219686 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-znkvp" Jul 01 18:42:57.178602 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:42:57.178574 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-wkbsg" Jul 01 18:43:04.216195 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:04.216161 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-9qxp2" Jul 01 18:43:46.882247 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.882211 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-jpg24"] Jul 01 18:43:46.884377 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.884354 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:46.888927 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.888900 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 18:43:46.889110 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.889087 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 18:43:46.889370 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.889353 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 01 18:43:46.889664 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.889426 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-htxbj\"" Jul 01 18:43:46.908724 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.908699 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-jpg24"] Jul 01 18:43:46.912672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.912646 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzs2d\" (UniqueName: \"kubernetes.io/projected/c857f9d9-6d3b-4206-8c19-a3021aac240f-kube-api-access-tzs2d\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:46.912766 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:46.912688 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.013488 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.013438 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzs2d\" (UniqueName: \"kubernetes.io/projected/c857f9d9-6d3b-4206-8c19-a3021aac240f-kube-api-access-tzs2d\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.013672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.013506 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.013672 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:43:47.013622 2554 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 01 18:43:47.013751 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:43:47.013686 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert podName:c857f9d9-6d3b-4206-8c19-a3021aac240f nodeName:}" failed. No retries permitted until 2026-07-01 18:43:47.513664909 +0000 UTC m=+386.116824790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert") pod "llmisvc-controller-manager-67d95fdc48-jpg24" (UID: "c857f9d9-6d3b-4206-8c19-a3021aac240f") : secret "llmisvc-webhook-server-cert" not found Jul 01 18:43:47.024554 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.024527 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzs2d\" (UniqueName: \"kubernetes.io/projected/c857f9d9-6d3b-4206-8c19-a3021aac240f-kube-api-access-tzs2d\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.518546 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.518503 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.520960 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.520939 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c857f9d9-6d3b-4206-8c19-a3021aac240f-cert\") pod \"llmisvc-controller-manager-67d95fdc48-jpg24\" (UID: \"c857f9d9-6d3b-4206-8c19-a3021aac240f\") " pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.797537 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.797422 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:47.937561 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:47.937530 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67d95fdc48-jpg24"] Jul 01 18:43:47.940599 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:43:47.940573 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc857f9d9_6d3b_4206_8c19_a3021aac240f.slice/crio-aee72c43a32e4e3f9dafd5b773febfeaa9252efc254cf1aeabe7063926c4391c WatchSource:0}: Error finding container aee72c43a32e4e3f9dafd5b773febfeaa9252efc254cf1aeabe7063926c4391c: Status 404 returned error can't find the container with id aee72c43a32e4e3f9dafd5b773febfeaa9252efc254cf1aeabe7063926c4391c Jul 01 18:43:48.490671 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:48.490631 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" event={"ID":"c857f9d9-6d3b-4206-8c19-a3021aac240f","Type":"ContainerStarted","Data":"aee72c43a32e4e3f9dafd5b773febfeaa9252efc254cf1aeabe7063926c4391c"} Jul 01 18:43:50.500757 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:50.500724 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" event={"ID":"c857f9d9-6d3b-4206-8c19-a3021aac240f","Type":"ContainerStarted","Data":"38beac96c064beb3114f1ce81e822a947007b3ab4e92ec5b9d1d2d94e508354e"} Jul 01 18:43:50.501128 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:50.500788 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:43:50.531313 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:43:50.531259 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" podStartSLOduration=2.88477902 podStartE2EDuration="4.531241315s" totalImagesPullingTime="1.646462295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:43:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:43:50.530474609 +0000 UTC m=+389.133634497" watchObservedRunningTime="2026-07-01 18:43:50.531241315 +0000 UTC m=+389.134401210" Jul 01 18:44:21.507081 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:21.507051 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67d95fdc48-jpg24" Jul 01 18:44:57.266880 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.266812 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6wklr"] Jul 01 18:44:57.270053 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.270037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.272899 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.272878 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-9mzfd\"" Jul 01 18:44:57.273193 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.273179 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jul 01 18:44:57.281873 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.281849 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6wklr"] Jul 01 18:44:57.413568 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.413539 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b5a3fc-731a-4fba-aba1-326005c49c6b-cert\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.413568 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.413570 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv2dq\" (UniqueName: \"kubernetes.io/projected/27b5a3fc-731a-4fba-aba1-326005c49c6b-kube-api-access-fv2dq\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.514579 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.514545 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b5a3fc-731a-4fba-aba1-326005c49c6b-cert\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.514579 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.514580 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv2dq\" (UniqueName: \"kubernetes.io/projected/27b5a3fc-731a-4fba-aba1-326005c49c6b-kube-api-access-fv2dq\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.517091 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.517018 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27b5a3fc-731a-4fba-aba1-326005c49c6b-cert\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.524216 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.524187 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv2dq\" (UniqueName: \"kubernetes.io/projected/27b5a3fc-731a-4fba-aba1-326005c49c6b-kube-api-access-fv2dq\") pod \"odh-model-controller-84b54b7f97-6wklr\" (UID: \"27b5a3fc-731a-4fba-aba1-326005c49c6b\") " pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.580150 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.580119 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:44:57.717328 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.717299 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-84b54b7f97-6wklr"] Jul 01 18:44:57.720782 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:44:57.720755 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27b5a3fc_731a_4fba_aba1_326005c49c6b.slice/crio-5af301f1b5945a91c64435dfafd2a2219d625d2428f8c8eae1920a83172473c8 WatchSource:0}: Error finding container 5af301f1b5945a91c64435dfafd2a2219d625d2428f8c8eae1920a83172473c8: Status 404 returned error can't find the container with id 5af301f1b5945a91c64435dfafd2a2219d625d2428f8c8eae1920a83172473c8 Jul 01 18:44:57.780928 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:44:57.780856 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-6wklr" event={"ID":"27b5a3fc-731a-4fba-aba1-326005c49c6b","Type":"ContainerStarted","Data":"5af301f1b5945a91c64435dfafd2a2219d625d2428f8c8eae1920a83172473c8"} Jul 01 18:45:00.798708 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:45:00.798669 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-84b54b7f97-6wklr" event={"ID":"27b5a3fc-731a-4fba-aba1-326005c49c6b","Type":"ContainerStarted","Data":"626f3889874f458c00acba19b660ae293f509bc94714a5183d5dd56efa9292aa"} Jul 01 18:45:00.799101 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:45:00.798794 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:45:00.818316 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:45:00.818217 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-84b54b7f97-6wklr" podStartSLOduration=0.984344823 podStartE2EDuration="3.818198595s" totalImagesPullingTime="2.833853772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:45:00.816727052 +0000 UTC m=+459.419886947" watchObservedRunningTime="2026-07-01 18:45:00.818198595 +0000 UTC m=+459.421358506" Jul 01 18:45:11.806986 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:45:11.806958 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-84b54b7f97-6wklr" Jul 01 18:47:21.794770 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:47:21.794742 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:47:21.795285 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:47:21.795062 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:48:04.443690 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.443656 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:04.446914 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.446896 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.449154 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.449124 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f2cdf-serving-cert\"" Jul 01 18:48:04.449279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.449179 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 18:48:04.449279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.449215 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qplkr\"" Jul 01 18:48:04.449279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.449220 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-f2cdf-kube-rbac-proxy-sar-config\"" Jul 01 18:48:04.456408 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.456388 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:04.594537 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.594507 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.594736 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.594660 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.696123 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.696026 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.696123 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.696077 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.696772 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.696749 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.698519 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.698502 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls\") pod \"switch-graph-f2cdf-545944cd54-5rxqp\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.757196 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.757158 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:04.890355 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.890328 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:04.893428 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:48:04.893399 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef20b52_23bf_4711_8ff0_91fcc81a715e.slice/crio-9686f58d0fd3fba6e21b1464ead3449ecb45284f20fda181948a0ce6f0623fd0 WatchSource:0}: Error finding container 9686f58d0fd3fba6e21b1464ead3449ecb45284f20fda181948a0ce6f0623fd0: Status 404 returned error can't find the container with id 9686f58d0fd3fba6e21b1464ead3449ecb45284f20fda181948a0ce6f0623fd0 Jul 01 18:48:04.895609 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:04.895590 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 18:48:05.582327 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:05.582293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" event={"ID":"cef20b52-23bf-4711-8ff0-91fcc81a715e","Type":"ContainerStarted","Data":"9686f58d0fd3fba6e21b1464ead3449ecb45284f20fda181948a0ce6f0623fd0"} Jul 01 18:48:08.598697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:08.598663 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" event={"ID":"cef20b52-23bf-4711-8ff0-91fcc81a715e","Type":"ContainerStarted","Data":"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb"} Jul 01 18:48:08.599108 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:08.598736 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:08.616720 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:08.616669 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podStartSLOduration=1.5370898020000001 podStartE2EDuration="4.616649157s" totalImagesPullingTime="3.079559355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:48:08.616440527 +0000 UTC m=+647.219600431" watchObservedRunningTime="2026-07-01 18:48:08.616649157 +0000 UTC m=+647.219809051" Jul 01 18:48:14.609914 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:14.609884 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:18.586128 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:18.586096 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:18.586558 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:18.586318 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" containerID="cri-o://8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb" gracePeriod=30 Jul 01 18:48:19.607285 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:19.607240 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:24.607412 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:24.607373 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:29.607764 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:29.607725 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:29.608128 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:29.607831 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:34.607396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:34.607360 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:39.607309 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:39.607269 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:44.607723 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:44.607675 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:48:48.733787 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.733760 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:48.779722 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.779692 2554 generic.go:356] "Generic (PLEG): container finished" podID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerID="8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb" exitCode=0 Jul 01 18:48:48.779877 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.779738 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" Jul 01 18:48:48.779877 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.779781 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" event={"ID":"cef20b52-23bf-4711-8ff0-91fcc81a715e","Type":"ContainerDied","Data":"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb"} Jul 01 18:48:48.779877 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.779864 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp" event={"ID":"cef20b52-23bf-4711-8ff0-91fcc81a715e","Type":"ContainerDied","Data":"9686f58d0fd3fba6e21b1464ead3449ecb45284f20fda181948a0ce6f0623fd0"} Jul 01 18:48:48.780007 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.779893 2554 scope.go:117] "RemoveContainer" containerID="8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb" Jul 01 18:48:48.789974 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.789941 2554 scope.go:117] "RemoveContainer" containerID="8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb" Jul 01 18:48:48.790172 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.790154 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls\") pod \"cef20b52-23bf-4711-8ff0-91fcc81a715e\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " Jul 01 18:48:48.790438 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:48:48.790400 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb\": container with ID starting with 8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb not found: ID does not exist" containerID="8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb" Jul 01 18:48:48.790552 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.790443 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb"} err="failed to get container status \"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb\": rpc error: code = NotFound desc = could not find container \"8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb\": container with ID starting with 8631e5e7cbb96b14e7b1d58fe1edc53cf032b98cbbdbeb146576e3b3cc479efb not found: ID does not exist" Jul 01 18:48:48.792223 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.792200 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cef20b52-23bf-4711-8ff0-91fcc81a715e" (UID: "cef20b52-23bf-4711-8ff0-91fcc81a715e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:48:48.891150 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.891084 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle\") pod \"cef20b52-23bf-4711-8ff0-91fcc81a715e\" (UID: \"cef20b52-23bf-4711-8ff0-91fcc81a715e\") " Jul 01 18:48:48.891267 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.891260 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cef20b52-23bf-4711-8ff0-91fcc81a715e-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:48:48.891394 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.891373 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cef20b52-23bf-4711-8ff0-91fcc81a715e" (UID: "cef20b52-23bf-4711-8ff0-91fcc81a715e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:48:48.992209 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:48.992175 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef20b52-23bf-4711-8ff0-91fcc81a715e-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:48:49.101593 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:49.101565 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:49.105635 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:49.105615 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-f2cdf-545944cd54-5rxqp"] Jul 01 18:48:49.824110 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:48:49.824077 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" path="/var/lib/kubelet/pods/cef20b52-23bf-4711-8ff0-91fcc81a715e/volumes" Jul 01 18:49:04.407857 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.407818 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:04.408356 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.408337 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" Jul 01 18:49:04.408440 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.408359 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" Jul 01 18:49:04.408528 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.408441 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef20b52-23bf-4711-8ff0-91fcc81a715e" containerName="switch-graph-f2cdf" Jul 01 18:49:04.411705 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.411683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:04.413840 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.413822 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 01 18:49:04.413935 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.413859 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 18:49:04.413994 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.413954 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qplkr\"" Jul 01 18:49:04.414147 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.414132 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 01 18:49:04.421252 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.421225 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:04.519151 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.519119 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:04.519306 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.519196 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:04.620230 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.620199 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:04.620383 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.620273 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:04.620434 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:49:04.620380 2554 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Jul 01 18:49:04.620497 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:49:04.620445 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls podName:0371b3ce-3a11-45f5-b918-a9363c53fc3c nodeName:}" failed. No retries permitted until 2026-07-01 18:49:05.120426707 +0000 UTC m=+703.723586596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls") pod "model-chainer-7565859b98-gvnv8" (UID: "0371b3ce-3a11-45f5-b918-a9363c53fc3c") : secret "model-chainer-serving-cert" not found Jul 01 18:49:04.620822 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:04.620804 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:05.124525 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.124493 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:05.126994 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.126958 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") pod \"model-chainer-7565859b98-gvnv8\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:05.323220 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.323192 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:05.452389 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.452364 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:05.454840 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:49:05.454808 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0371b3ce_3a11_45f5_b918_a9363c53fc3c.slice/crio-50b9f061cb51c327367ec91059744d0f5e30068041cf94b04358386e80788b88 WatchSource:0}: Error finding container 50b9f061cb51c327367ec91059744d0f5e30068041cf94b04358386e80788b88: Status 404 returned error can't find the container with id 50b9f061cb51c327367ec91059744d0f5e30068041cf94b04358386e80788b88 Jul 01 18:49:05.861500 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.861449 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" event={"ID":"0371b3ce-3a11-45f5-b918-a9363c53fc3c","Type":"ContainerStarted","Data":"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7"} Jul 01 18:49:05.861665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.861518 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" event={"ID":"0371b3ce-3a11-45f5-b918-a9363c53fc3c","Type":"ContainerStarted","Data":"50b9f061cb51c327367ec91059744d0f5e30068041cf94b04358386e80788b88"} Jul 01 18:49:05.861665 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.861558 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:05.882087 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:05.882044 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podStartSLOduration=1.882028955 podStartE2EDuration="1.882028955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:49:05.879774251 +0000 UTC m=+704.482934144" watchObservedRunningTime="2026-07-01 18:49:05.882028955 +0000 UTC m=+704.485188888" Jul 01 18:49:11.871669 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:11.871626 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:14.511311 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:14.511230 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:14.511735 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:14.511411 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" containerID="cri-o://69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7" gracePeriod=30 Jul 01 18:49:16.869813 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:16.869771 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:18.815798 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.815769 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:49:18.818227 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.818210 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:18.820541 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.820521 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-9857d-serving-cert\"" Jul 01 18:49:18.820664 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.820524 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-9857d-kube-rbac-proxy-sar-config\"" Jul 01 18:49:18.827023 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.827005 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:49:18.939965 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.939932 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:18.940117 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:18.939976 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.041333 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.041303 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.041498 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.041344 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.041935 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.041906 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.043652 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.043633 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls\") pod \"switch-graph-9857d-698d9d6767-gzxql\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.129183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.129100 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.259424 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.259367 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:49:19.261599 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:49:19.261563 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b236198_ea57_4094_814f_197d7b3adcf3.slice/crio-5bb9263e8469df128dffe537330fe9c4b9aaea7d22a5a30458a718b6ba61d3b6 WatchSource:0}: Error finding container 5bb9263e8469df128dffe537330fe9c4b9aaea7d22a5a30458a718b6ba61d3b6: Status 404 returned error can't find the container with id 5bb9263e8469df128dffe537330fe9c4b9aaea7d22a5a30458a718b6ba61d3b6 Jul 01 18:49:19.923274 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.923241 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" event={"ID":"0b236198-ea57-4094-814f-197d7b3adcf3","Type":"ContainerStarted","Data":"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d"} Jul 01 18:49:19.923672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.923289 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" event={"ID":"0b236198-ea57-4094-814f-197d7b3adcf3","Type":"ContainerStarted","Data":"5bb9263e8469df128dffe537330fe9c4b9aaea7d22a5a30458a718b6ba61d3b6"} Jul 01 18:49:19.923672 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.923442 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:19.942577 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:19.942531 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podStartSLOduration=1.942516961 podStartE2EDuration="1.942516961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:49:19.940715146 +0000 UTC m=+718.543875041" watchObservedRunningTime="2026-07-01 18:49:19.942516961 +0000 UTC m=+718.545676854" Jul 01 18:49:21.870020 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:21.869988 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:25.935476 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:25.935435 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:49:26.869972 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:26.869929 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:26.870161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:26.870039 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:31.870676 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:31.870631 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:36.869440 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:36.869403 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:41.869720 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:41.869677 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:49:44.657240 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.657217 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:44.660127 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.660110 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle\") pod \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " Jul 01 18:49:44.660183 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.660148 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") pod \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\" (UID: \"0371b3ce-3a11-45f5-b918-a9363c53fc3c\") " Jul 01 18:49:44.660422 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.660400 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0371b3ce-3a11-45f5-b918-a9363c53fc3c" (UID: "0371b3ce-3a11-45f5-b918-a9363c53fc3c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:49:44.662095 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.662070 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0371b3ce-3a11-45f5-b918-a9363c53fc3c" (UID: "0371b3ce-3a11-45f5-b918-a9363c53fc3c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:49:44.761429 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.761393 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0371b3ce-3a11-45f5-b918-a9363c53fc3c-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:49:44.761429 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:44.761421 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0371b3ce-3a11-45f5-b918-a9363c53fc3c-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:49:45.032413 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.032383 2554 generic.go:356] "Generic (PLEG): container finished" podID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerID="69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7" exitCode=0 Jul 01 18:49:45.032583 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.032428 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" Jul 01 18:49:45.032583 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.032485 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" event={"ID":"0371b3ce-3a11-45f5-b918-a9363c53fc3c","Type":"ContainerDied","Data":"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7"} Jul 01 18:49:45.032583 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.032532 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8" event={"ID":"0371b3ce-3a11-45f5-b918-a9363c53fc3c","Type":"ContainerDied","Data":"50b9f061cb51c327367ec91059744d0f5e30068041cf94b04358386e80788b88"} Jul 01 18:49:45.032583 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.032555 2554 scope.go:117] "RemoveContainer" containerID="69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7" Jul 01 18:49:45.043646 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.043612 2554 scope.go:117] "RemoveContainer" containerID="69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7" Jul 01 18:49:45.044063 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:49:45.044044 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7\": container with ID starting with 69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7 not found: ID does not exist" containerID="69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7" Jul 01 18:49:45.044126 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.044072 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7"} err="failed to get container status \"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7\": rpc error: code = NotFound desc = could not find container \"69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7\": container with ID starting with 69d3cb5e744484b2e9158bceca738cbb591c9dd970e7618083d21fe1a84750b7 not found: ID does not exist" Jul 01 18:49:45.057017 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.056991 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:45.059645 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.059623 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7565859b98-gvnv8"] Jul 01 18:49:45.825116 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:49:45.825082 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" path="/var/lib/kubelet/pods/0371b3ce-3a11-45f5-b918-a9363c53fc3c/volumes" Jul 01 18:50:14.702224 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.702189 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:50:14.702777 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.702757 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" Jul 01 18:50:14.702849 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.702780 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" Jul 01 18:50:14.702960 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.702945 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0371b3ce-3a11-45f5-b918-a9363c53fc3c" containerName="model-chainer" Jul 01 18:50:14.705047 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.705027 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.707372 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.707352 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-eb211-kube-rbac-proxy-sar-config\"" Jul 01 18:50:14.707444 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.707389 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-eb211-serving-cert\"" Jul 01 18:50:14.717411 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.717392 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:50:14.794827 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.794792 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.794963 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.794847 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.896021 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.895984 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.896021 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.896028 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.896659 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.896633 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:14.898342 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:14.898322 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls\") pod \"sequence-graph-eb211-648cf9b67b-h6dpc\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:15.014497 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:15.014473 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:15.141207 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:15.141173 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:50:15.144261 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:50:15.144236 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6acf93_bdda_4f40_9e82_3ecafb59c829.slice/crio-7728cedd22e1a2c7531bb8f47b4a4354d14af4c6bd7e6e0c3ca9175d61160dc0 WatchSource:0}: Error finding container 7728cedd22e1a2c7531bb8f47b4a4354d14af4c6bd7e6e0c3ca9175d61160dc0: Status 404 returned error can't find the container with id 7728cedd22e1a2c7531bb8f47b4a4354d14af4c6bd7e6e0c3ca9175d61160dc0 Jul 01 18:50:15.167093 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:15.167067 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" event={"ID":"ce6acf93-bdda-4f40-9e82-3ecafb59c829","Type":"ContainerStarted","Data":"7728cedd22e1a2c7531bb8f47b4a4354d14af4c6bd7e6e0c3ca9175d61160dc0"} Jul 01 18:50:16.173674 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:16.173641 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" event={"ID":"ce6acf93-bdda-4f40-9e82-3ecafb59c829","Type":"ContainerStarted","Data":"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178"} Jul 01 18:50:16.174161 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:16.173704 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:50:16.192332 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:16.192287 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podStartSLOduration=2.192271055 podStartE2EDuration="2.192271055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:50:16.190244827 +0000 UTC m=+774.793404722" watchObservedRunningTime="2026-07-01 18:50:16.192271055 +0000 UTC m=+774.795430984" Jul 01 18:50:22.184666 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:50:22.184576 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:52:21.819387 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:52:21.819357 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:52:21.821430 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:52:21.821397 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:57:21.850443 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:21.850416 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:57:21.853165 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:21.853140 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 18:57:33.439891 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:33.439857 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:57:33.440321 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:33.440058 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" containerID="cri-o://4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d" gracePeriod=30 Jul 01 18:57:35.933862 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:35.933816 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:57:40.934005 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:40.933961 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:57:45.933589 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:45.933548 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:57:45.934054 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:45.933657 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:57:50.934620 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:50.934579 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:57:55.934411 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:57:55.934365 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:00.933632 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:00.933592 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:04.087085 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.087064 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:58:04.208252 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.208174 2554 generic.go:356] "Generic (PLEG): container finished" podID="0b236198-ea57-4094-814f-197d7b3adcf3" containerID="4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d" exitCode=0 Jul 01 18:58:04.208252 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.208214 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" event={"ID":"0b236198-ea57-4094-814f-197d7b3adcf3","Type":"ContainerDied","Data":"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d"} Jul 01 18:58:04.208252 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.208217 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" Jul 01 18:58:04.208505 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.208261 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql" event={"ID":"0b236198-ea57-4094-814f-197d7b3adcf3","Type":"ContainerDied","Data":"5bb9263e8469df128dffe537330fe9c4b9aaea7d22a5a30458a718b6ba61d3b6"} Jul 01 18:58:04.208505 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.208284 2554 scope.go:117] "RemoveContainer" containerID="4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d" Jul 01 18:58:04.216482 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.216440 2554 scope.go:117] "RemoveContainer" containerID="4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d" Jul 01 18:58:04.216754 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:58:04.216733 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d\": container with ID starting with 4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d not found: ID does not exist" containerID="4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d" Jul 01 18:58:04.216820 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.216764 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d"} err="failed to get container status \"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d\": rpc error: code = NotFound desc = could not find container \"4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d\": container with ID starting with 4bcf839a6ca140d02fdca760b8fa5d73ef4209d5af14111a6a4a185aa97bf10d not found: ID does not exist" Jul 01 18:58:04.223406 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.223387 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls\") pod \"0b236198-ea57-4094-814f-197d7b3adcf3\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " Jul 01 18:58:04.223535 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.223468 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle\") pod \"0b236198-ea57-4094-814f-197d7b3adcf3\" (UID: \"0b236198-ea57-4094-814f-197d7b3adcf3\") " Jul 01 18:58:04.223807 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.223786 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0b236198-ea57-4094-814f-197d7b3adcf3" (UID: "0b236198-ea57-4094-814f-197d7b3adcf3"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:58:04.225339 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.225310 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0b236198-ea57-4094-814f-197d7b3adcf3" (UID: "0b236198-ea57-4094-814f-197d7b3adcf3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:58:04.324903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.324875 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0b236198-ea57-4094-814f-197d7b3adcf3-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:58:04.324903 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.324899 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b236198-ea57-4094-814f-197d7b3adcf3-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:58:04.530475 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.530421 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:58:04.535267 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:04.535243 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-9857d-698d9d6767-gzxql"] Jul 01 18:58:05.824622 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:05.824589 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" path="/var/lib/kubelet/pods/0b236198-ea57-4094-814f-197d7b3adcf3/volumes" Jul 01 18:58:29.321273 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:29.321241 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:58:29.321768 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:29.321436 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" containerID="cri-o://4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178" gracePeriod=30 Jul 01 18:58:32.182933 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:32.182891 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:33.676983 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.676944 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:58:33.677363 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.677266 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" Jul 01 18:58:33.677363 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.677277 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" Jul 01 18:58:33.677363 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.677347 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b236198-ea57-4094-814f-197d7b3adcf3" containerName="switch-graph-9857d" Jul 01 18:58:33.681629 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.681612 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.684041 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.684020 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c059b-serving-cert\"" Jul 01 18:58:33.684501 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.684486 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-c059b-kube-rbac-proxy-sar-config\"" Jul 01 18:58:33.689265 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.689244 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:58:33.774792 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.774759 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.774974 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.774824 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.875736 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.875708 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.875929 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.875770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.876370 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.876344 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.878221 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.878200 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls\") pod \"ensemble-graph-c059b-b4df54dc8-fppfl\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:33.992554 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:33.992443 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:34.126487 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:34.126445 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:58:34.128233 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:58:34.128209 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ecbd2a_6ea7_41de_850a_369c27f35b9a.slice/crio-75547f6ff6f1d91db4bdbc2505cf1afee2134844e85a8bd6e39f5cd8400e71d8 WatchSource:0}: Error finding container 75547f6ff6f1d91db4bdbc2505cf1afee2134844e85a8bd6e39f5cd8400e71d8: Status 404 returned error can't find the container with id 75547f6ff6f1d91db4bdbc2505cf1afee2134844e85a8bd6e39f5cd8400e71d8 Jul 01 18:58:34.342277 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:34.342242 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" event={"ID":"e8ecbd2a-6ea7-41de-850a-369c27f35b9a","Type":"ContainerStarted","Data":"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26"} Jul 01 18:58:34.342482 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:34.342293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" event={"ID":"e8ecbd2a-6ea7-41de-850a-369c27f35b9a","Type":"ContainerStarted","Data":"75547f6ff6f1d91db4bdbc2505cf1afee2134844e85a8bd6e39f5cd8400e71d8"} Jul 01 18:58:34.342482 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:34.342381 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:34.362146 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:34.362101 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podStartSLOduration=1.362087158 podStartE2EDuration="1.362087158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:58:34.361444544 +0000 UTC m=+1272.964604430" watchObservedRunningTime="2026-07-01 18:58:34.362087158 +0000 UTC m=+1272.965247080" Jul 01 18:58:37.182887 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:37.182849 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:40.353208 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:40.353173 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:42.182623 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:42.182588 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:42.183024 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:42.182691 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:58:43.746870 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:43.746840 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:58:43.747254 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:43.747014 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" containerID="cri-o://bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26" gracePeriod=30 Jul 01 18:58:45.351279 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:45.351238 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:47.182418 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:47.182381 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:50.350955 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:50.350920 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:52.182023 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:52.181982 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:55.351477 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:55.351409 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:55.351940 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:55.351553 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:58:57.181991 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:57.181949 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:58:59.456391 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.456356 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:58:59.456782 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.456598 2554 generic.go:356] "Generic (PLEG): container finished" podID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerID="4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178" exitCode=0 Jul 01 18:58:59.456782 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.456681 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" event={"ID":"ce6acf93-bdda-4f40-9e82-3ecafb59c829","Type":"ContainerDied","Data":"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178"} Jul 01 18:58:59.456782 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.456749 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" event={"ID":"ce6acf93-bdda-4f40-9e82-3ecafb59c829","Type":"ContainerDied","Data":"7728cedd22e1a2c7531bb8f47b4a4354d14af4c6bd7e6e0c3ca9175d61160dc0"} Jul 01 18:58:59.456782 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.456778 2554 scope.go:117] "RemoveContainer" containerID="4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178" Jul 01 18:58:59.465013 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.464996 2554 scope.go:117] "RemoveContainer" containerID="4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178" Jul 01 18:58:59.465227 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:58:59.465206 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178\": container with ID starting with 4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178 not found: ID does not exist" containerID="4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178" Jul 01 18:58:59.465291 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.465240 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178"} err="failed to get container status \"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178\": rpc error: code = NotFound desc = could not find container \"4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178\": container with ID starting with 4ced42acf811446acdf455227c0e9244139280e461a812dad5bafcae5c039178 not found: ID does not exist" Jul 01 18:58:59.497043 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.497014 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls\") pod \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " Jul 01 18:58:59.497185 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.497084 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle\") pod \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\" (UID: \"ce6acf93-bdda-4f40-9e82-3ecafb59c829\") " Jul 01 18:58:59.497421 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.497399 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ce6acf93-bdda-4f40-9e82-3ecafb59c829" (UID: "ce6acf93-bdda-4f40-9e82-3ecafb59c829"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:58:59.499009 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.498987 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ce6acf93-bdda-4f40-9e82-3ecafb59c829" (UID: "ce6acf93-bdda-4f40-9e82-3ecafb59c829"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:58:59.597884 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.597806 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ce6acf93-bdda-4f40-9e82-3ecafb59c829-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:58:59.597884 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:58:59.597832 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce6acf93-bdda-4f40-9e82-3ecafb59c829-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:59:00.350723 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:00.350686 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:59:00.461779 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:00.461755 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc" Jul 01 18:59:00.480497 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:00.480469 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:59:00.489926 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:00.489904 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-eb211-648cf9b67b-h6dpc"] Jul 01 18:59:01.824884 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:01.824848 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" path="/var/lib/kubelet/pods/ce6acf93-bdda-4f40-9e82-3ecafb59c829/volumes" Jul 01 18:59:05.350792 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:05.350747 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:59:10.351346 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:10.351303 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 18:59:13.924660 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:13.924630 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:59:14.020717 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.020612 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls\") pod \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " Jul 01 18:59:14.020885 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.020719 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle\") pod \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\" (UID: \"e8ecbd2a-6ea7-41de-850a-369c27f35b9a\") " Jul 01 18:59:14.021057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.021029 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e8ecbd2a-6ea7-41de-850a-369c27f35b9a" (UID: "e8ecbd2a-6ea7-41de-850a-369c27f35b9a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 18:59:14.022697 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.022673 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e8ecbd2a-6ea7-41de-850a-369c27f35b9a" (UID: "e8ecbd2a-6ea7-41de-850a-369c27f35b9a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 18:59:14.122356 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.122313 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:59:14.122356 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.122353 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e8ecbd2a-6ea7-41de-850a-369c27f35b9a-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 18:59:14.527360 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.527326 2554 generic.go:356] "Generic (PLEG): container finished" podID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerID="bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26" exitCode=137 Jul 01 18:59:14.527541 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.527369 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" Jul 01 18:59:14.527541 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.527407 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" event={"ID":"e8ecbd2a-6ea7-41de-850a-369c27f35b9a","Type":"ContainerDied","Data":"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26"} Jul 01 18:59:14.527541 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.527480 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl" event={"ID":"e8ecbd2a-6ea7-41de-850a-369c27f35b9a","Type":"ContainerDied","Data":"75547f6ff6f1d91db4bdbc2505cf1afee2134844e85a8bd6e39f5cd8400e71d8"} Jul 01 18:59:14.527541 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.527505 2554 scope.go:117] "RemoveContainer" containerID="bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26" Jul 01 18:59:14.535891 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.535874 2554 scope.go:117] "RemoveContainer" containerID="bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26" Jul 01 18:59:14.536137 ip-10-0-139-178 kubenswrapper[2554]: E0701 18:59:14.536119 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26\": container with ID starting with bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26 not found: ID does not exist" containerID="bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26" Jul 01 18:59:14.536189 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.536146 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26"} err="failed to get container status \"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26\": rpc error: code = NotFound desc = could not find container \"bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26\": container with ID starting with bc2c82903daa9a17ea2570ce5583e2f95fabb5688648da9de91f5bcc49c1df26 not found: ID does not exist" Jul 01 18:59:14.551943 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.551912 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:59:14.556045 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:14.556023 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-c059b-b4df54dc8-fppfl"] Jul 01 18:59:15.824278 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:15.824247 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" path="/var/lib/kubelet/pods/e8ecbd2a-6ea7-41de-850a-369c27f35b9a/volumes" Jul 01 18:59:39.511152 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511119 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511481 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511492 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511525 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511531 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511575 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8ecbd2a-6ea7-41de-850a-369c27f35b9a" containerName="ensemble-graph-c059b" Jul 01 18:59:39.511634 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.511584 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce6acf93-bdda-4f40-9e82-3ecafb59c829" containerName="sequence-graph-eb211" Jul 01 18:59:39.515921 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.515901 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.518021 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.517999 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-550f3-serving-cert\"" Jul 01 18:59:39.518134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.518095 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-550f3-kube-rbac-proxy-sar-config\"" Jul 01 18:59:39.518134 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.518104 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qplkr\"" Jul 01 18:59:39.518247 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.518138 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 18:59:39.522958 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.522902 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 18:59:39.644359 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.644329 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.644545 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.644378 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.745829 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.745789 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.745999 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.745843 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.746444 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.746414 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.748226 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.748206 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls\") pod \"sequence-graph-550f3-686769c68-t5pr2\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.826624 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.826543 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:39.961125 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:39.961048 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 18:59:39.963924 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:59:39.963897 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8d03ab_9bb3_4ac6_8794_aa4b47db3fb4.slice/crio-1d8e8997673f2259564d36e2604e61ba078716d6e2a1bab2ab3294b640f1cd24 WatchSource:0}: Error finding container 1d8e8997673f2259564d36e2604e61ba078716d6e2a1bab2ab3294b640f1cd24: Status 404 returned error can't find the container with id 1d8e8997673f2259564d36e2604e61ba078716d6e2a1bab2ab3294b640f1cd24 Jul 01 18:59:40.650923 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:40.650887 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" event={"ID":"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4","Type":"ContainerStarted","Data":"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b"} Jul 01 18:59:40.651396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:40.650945 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" event={"ID":"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4","Type":"ContainerStarted","Data":"1d8e8997673f2259564d36e2604e61ba078716d6e2a1bab2ab3294b640f1cd24"} Jul 01 18:59:40.651396 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:40.651003 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:40.669008 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:40.668910 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podStartSLOduration=1.6688937529999999 podStartE2EDuration="1.668893753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:59:40.66852133 +0000 UTC m=+1339.271681225" watchObservedRunningTime="2026-07-01 18:59:40.668893753 +0000 UTC m=+1339.272053650" Jul 01 18:59:43.973057 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:43.973016 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 18:59:43.977785 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:43.977762 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:43.980289 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:43.980267 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-af621-kube-rbac-proxy-sar-config\"" Jul 01 18:59:43.980358 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:43.980267 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-af621-serving-cert\"" Jul 01 18:59:43.983130 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:43.983106 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 18:59:44.083922 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.083882 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.084093 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.083989 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.184568 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.184538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.184689 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.184603 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.185321 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.185295 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.186983 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.186964 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls\") pod \"ensemble-graph-af621-7d4fd998f4-sv5l4\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.290082 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.290011 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.422791 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.422754 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 18:59:44.424855 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:59:44.424820 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5c9b1a_46ff_4794_985b_6e2a883239e0.slice/crio-6e11e03e1958e23614a6cb965510d9aef49dfc9dd5f07a9142d7e265be0ac267 WatchSource:0}: Error finding container 6e11e03e1958e23614a6cb965510d9aef49dfc9dd5f07a9142d7e265be0ac267: Status 404 returned error can't find the container with id 6e11e03e1958e23614a6cb965510d9aef49dfc9dd5f07a9142d7e265be0ac267 Jul 01 18:59:44.669496 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.669370 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" event={"ID":"ba5c9b1a-46ff-4794-985b-6e2a883239e0","Type":"ContainerStarted","Data":"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07"} Jul 01 18:59:44.669496 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.669425 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" event={"ID":"ba5c9b1a-46ff-4794-985b-6e2a883239e0","Type":"ContainerStarted","Data":"6e11e03e1958e23614a6cb965510d9aef49dfc9dd5f07a9142d7e265be0ac267"} Jul 01 18:59:44.669727 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.669559 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:44.689258 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:44.689208 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podStartSLOduration=1.6891921189999999 podStartE2EDuration="1.689192119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:59:44.688821439 +0000 UTC m=+1343.291981333" watchObservedRunningTime="2026-07-01 18:59:44.689192119 +0000 UTC m=+1343.292352012" Jul 01 18:59:46.662614 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:46.662580 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 18:59:49.792671 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.792637 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68ffd9f4d6-w7x85"] Jul 01 18:59:49.796206 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.796178 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.810682 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.810647 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffd9f4d6-w7x85"] Jul 01 18:59:49.825472 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825431 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-service-ca\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825609 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825525 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-trusted-ca-bundle\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825609 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825552 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-oauth-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825609 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825601 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825712 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825619 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-console-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825712 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825643 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-oauth-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.825773 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.825701 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwj6x\" (UniqueName: \"kubernetes.io/projected/e40a8f1e-2021-4625-8fe4-8186458b1318-kube-api-access-vwj6x\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926394 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926354 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwj6x\" (UniqueName: \"kubernetes.io/projected/e40a8f1e-2021-4625-8fe4-8186458b1318-kube-api-access-vwj6x\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926608 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-service-ca\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926608 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926510 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-trusted-ca-bundle\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926608 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-oauth-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926608 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926578 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926608 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926598 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-console-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.926858 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.926626 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-oauth-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.927212 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.927187 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-oauth-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.927334 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.927218 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-service-ca\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.927378 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.927334 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-console-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.927378 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.927355 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e40a8f1e-2021-4625-8fe4-8186458b1318-trusted-ca-bundle\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.929574 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.929546 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-serving-cert\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.929574 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.929564 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e40a8f1e-2021-4625-8fe4-8186458b1318-console-oauth-config\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:49.938675 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:49.938654 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwj6x\" (UniqueName: \"kubernetes.io/projected/e40a8f1e-2021-4625-8fe4-8186458b1318-kube-api-access-vwj6x\") pod \"console-68ffd9f4d6-w7x85\" (UID: \"e40a8f1e-2021-4625-8fe4-8186458b1318\") " pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:50.105239 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.105153 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 18:59:50.245689 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.245648 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffd9f4d6-w7x85"] Jul 01 18:59:50.248724 ip-10-0-139-178 kubenswrapper[2554]: W0701 18:59:50.248700 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40a8f1e_2021_4625_8fe4_8186458b1318.slice/crio-05852fcd9335decedbbcfdd520fc4cc68f419ffbac273d8ca71d5174aa09f61f WatchSource:0}: Error finding container 05852fcd9335decedbbcfdd520fc4cc68f419ffbac273d8ca71d5174aa09f61f: Status 404 returned error can't find the container with id 05852fcd9335decedbbcfdd520fc4cc68f419ffbac273d8ca71d5174aa09f61f Jul 01 18:59:50.679251 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.679223 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 18:59:50.698004 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.697977 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffd9f4d6-w7x85" event={"ID":"e40a8f1e-2021-4625-8fe4-8186458b1318","Type":"ContainerStarted","Data":"f9efda4c5c97b90580100f341c7d79fe106ba5524ba275049c2c0d9ce1c1818f"} Jul 01 18:59:50.698174 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.698053 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffd9f4d6-w7x85" event={"ID":"e40a8f1e-2021-4625-8fe4-8186458b1318","Type":"ContainerStarted","Data":"05852fcd9335decedbbcfdd520fc4cc68f419ffbac273d8ca71d5174aa09f61f"} Jul 01 18:59:50.729661 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:50.729611 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68ffd9f4d6-w7x85" podStartSLOduration=1.72959188 podStartE2EDuration="1.72959188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 18:59:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:59:50.728965816 +0000 UTC m=+1349.332125711" watchObservedRunningTime="2026-07-01 18:59:50.72959188 +0000 UTC m=+1349.332751775" Jul 01 18:59:53.635930 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:53.635895 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 18:59:53.636340 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:53.636108 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" containerID="cri-o://0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b" gracePeriod=30 Jul 01 18:59:56.659939 ip-10-0-139-178 kubenswrapper[2554]: I0701 18:59:56.659894 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:00.105836 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:00.105787 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 19:00:00.106272 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:00.105861 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 19:00:00.110935 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:00.110912 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 19:00:00.742769 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:00.742742 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68ffd9f4d6-w7x85" Jul 01 19:00:00.795099 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:00.795071 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 19:00:01.659205 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:01.659169 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:06.659550 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:06.659506 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:06.659905 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:06.659605 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 19:00:11.659502 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:11.659435 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:16.659160 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:16.659117 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:21.659168 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:21.659133 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:00:23.788780 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.788757 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 19:00:23.842260 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.842222 2554 generic.go:356] "Generic (PLEG): container finished" podID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerID="0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b" exitCode=0 Jul 01 19:00:23.842411 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.842272 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" Jul 01 19:00:23.842411 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.842276 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" event={"ID":"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4","Type":"ContainerDied","Data":"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b"} Jul 01 19:00:23.842411 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.842327 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2" event={"ID":"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4","Type":"ContainerDied","Data":"1d8e8997673f2259564d36e2604e61ba078716d6e2a1bab2ab3294b640f1cd24"} Jul 01 19:00:23.842411 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.842359 2554 scope.go:117] "RemoveContainer" containerID="0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b" Jul 01 19:00:23.850655 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.850639 2554 scope.go:117] "RemoveContainer" containerID="0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b" Jul 01 19:00:23.850912 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:00:23.850894 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b\": container with ID starting with 0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b not found: ID does not exist" containerID="0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b" Jul 01 19:00:23.850957 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.850921 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b"} err="failed to get container status \"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b\": rpc error: code = NotFound desc = could not find container \"0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b\": container with ID starting with 0da0a052bd5b9b1e1b8cb16f3bcf01a98d6bfa914083d0a8a83f09f39b0bde0b not found: ID does not exist" Jul 01 19:00:23.932038 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.931969 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls\") pod \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " Jul 01 19:00:23.932154 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.932074 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle\") pod \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\" (UID: \"9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4\") " Jul 01 19:00:23.932426 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.932405 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" (UID: "9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:00:23.934088 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:23.934070 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" (UID: "9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:00:24.033219 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:24.033188 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:24.033219 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:24.033214 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:24.165639 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:24.165610 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 19:00:24.169325 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:24.169298 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-550f3-686769c68-t5pr2"] Jul 01 19:00:25.816202 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:25.816141 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-549b475fd4-hj6tb" podUID="895c7176-10e1-4112-bde2-63ba4542031f" containerName="console" containerID="cri-o://e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de" gracePeriod=15 Jul 01 19:00:25.824775 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:25.824750 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" path="/var/lib/kubelet/pods/9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4/volumes" Jul 01 19:00:26.068246 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.068191 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 19:00:26.151357 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151327 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151551 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151374 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151551 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151405 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151551 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151435 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151551 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151496 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151551 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151537 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lh8s\" (UniqueName: \"kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151823 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151559 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert\") pod \"895c7176-10e1-4112-bde2-63ba4542031f\" (UID: \"895c7176-10e1-4112-bde2-63ba4542031f\") " Jul 01 19:00:26.151906 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151876 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca" (OuterVolumeSpecName: "service-ca") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:00:26.151957 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151900 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config" (OuterVolumeSpecName: "console-config") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:00:26.151957 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151904 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:00:26.151957 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.151941 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:00:26.153583 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.153555 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:00:26.153690 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.153637 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:00:26.153690 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.153641 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s" (OuterVolumeSpecName: "kube-api-access-4lh8s") pod "895c7176-10e1-4112-bde2-63ba4542031f" (UID: "895c7176-10e1-4112-bde2-63ba4542031f"). InnerVolumeSpecName "kube-api-access-4lh8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 19:00:26.253171 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253139 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-trusted-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253171 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253165 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lh8s\" (UniqueName: \"kubernetes.io/projected/895c7176-10e1-4112-bde2-63ba4542031f-kube-api-access-4lh8s\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253171 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253174 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253386 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253183 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-console-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253386 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253192 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/895c7176-10e1-4112-bde2-63ba4542031f-console-oauth-config\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253386 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253200 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-service-ca\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.253386 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.253208 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/895c7176-10e1-4112-bde2-63ba4542031f-oauth-serving-cert\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:00:26.857268 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.857227 2554 generic.go:356] "Generic (PLEG): container finished" podID="895c7176-10e1-4112-bde2-63ba4542031f" containerID="e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de" exitCode=0 Jul 01 19:00:26.857682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.857277 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549b475fd4-hj6tb" Jul 01 19:00:26.857682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.857315 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b475fd4-hj6tb" event={"ID":"895c7176-10e1-4112-bde2-63ba4542031f","Type":"ContainerDied","Data":"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de"} Jul 01 19:00:26.857682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.857366 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549b475fd4-hj6tb" event={"ID":"895c7176-10e1-4112-bde2-63ba4542031f","Type":"ContainerDied","Data":"c297e12b4079686d8477651d543229166e90b6a0ef620c4c37095c58db2b9480"} Jul 01 19:00:26.857682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.857386 2554 scope.go:117] "RemoveContainer" containerID="e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de" Jul 01 19:00:26.866169 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.866153 2554 scope.go:117] "RemoveContainer" containerID="e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de" Jul 01 19:00:26.866503 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:00:26.866449 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de\": container with ID starting with e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de not found: ID does not exist" containerID="e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de" Jul 01 19:00:26.866503 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.866492 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de"} err="failed to get container status \"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de\": rpc error: code = NotFound desc = could not find container \"e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de\": container with ID starting with e474b6a5c4167d281edf94db2a603b6879e3d3d370fd3edcceec0a6e041aa9de not found: ID does not exist" Jul 01 19:00:26.879979 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.879958 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 19:00:26.883985 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:26.883967 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-549b475fd4-hj6tb"] Jul 01 19:00:27.825809 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:27.825775 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="895c7176-10e1-4112-bde2-63ba4542031f" path="/var/lib/kubelet/pods/895c7176-10e1-4112-bde2-63ba4542031f/volumes" Jul 01 19:00:53.888728 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.888691 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889035 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889046 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889064 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895c7176-10e1-4112-bde2-63ba4542031f" containerName="console" Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889070 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="895c7176-10e1-4112-bde2-63ba4542031f" containerName="console" Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889118 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8d03ab-9bb3-4ac6-8794-aa4b47db3fb4" containerName="sequence-graph-550f3" Jul 01 19:00:53.889135 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.889128 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="895c7176-10e1-4112-bde2-63ba4542031f" containerName="console" Jul 01 19:00:53.892039 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.892023 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:53.894644 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.894623 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3262-kube-rbac-proxy-sar-config\"" Jul 01 19:00:53.894897 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.894879 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-c3262-serving-cert\"" Jul 01 19:00:53.903918 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.903897 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:00:53.977512 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.977479 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:53.977676 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:53.977543 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.078759 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.078719 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.078927 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.078804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.079526 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.079508 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.081343 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.081322 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls\") pod \"sequence-graph-c3262-557db7f5b8-g279b\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.202653 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.202566 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.341428 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.341392 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:00:54.344394 ip-10-0-139-178 kubenswrapper[2554]: W0701 19:00:54.344368 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a0c79a_b5f5_43ef_a3db_b82cca330ffa.slice/crio-039d8615fb352089bf59b5f16b5a1d1c6aa15a00b3f8a625dab4dc2640899d00 WatchSource:0}: Error finding container 039d8615fb352089bf59b5f16b5a1d1c6aa15a00b3f8a625dab4dc2640899d00: Status 404 returned error can't find the container with id 039d8615fb352089bf59b5f16b5a1d1c6aa15a00b3f8a625dab4dc2640899d00 Jul 01 19:00:54.979971 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.979936 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" event={"ID":"12a0c79a-b5f5-43ef-a3db-b82cca330ffa","Type":"ContainerStarted","Data":"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17"} Jul 01 19:00:54.980353 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.980026 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:00:54.980353 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:54.980045 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" event={"ID":"12a0c79a-b5f5-43ef-a3db-b82cca330ffa","Type":"ContainerStarted","Data":"039d8615fb352089bf59b5f16b5a1d1c6aa15a00b3f8a625dab4dc2640899d00"} Jul 01 19:00:55.002066 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:00:55.002015 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podStartSLOduration=2.001999641 podStartE2EDuration="2.001999641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:00:55.001617349 +0000 UTC m=+1413.604777243" watchObservedRunningTime="2026-07-01 19:00:55.001999641 +0000 UTC m=+1413.605159528" Jul 01 19:01:00.990223 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:01:00.990195 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:02:21.877577 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:02:21.877547 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:02:21.880767 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:02:21.880749 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:07:21.903714 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:07:21.903687 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:07:21.908047 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:07:21.908023 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:07:58.704197 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:07:58.704160 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 19:07:58.705167 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:07:58.704402 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" containerID="cri-o://7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07" gracePeriod=30 Jul 01 19:08:00.678491 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:00.678435 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:05.677963 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:05.677921 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:10.678384 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:10.678334 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:10.678794 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:10.678447 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 19:08:15.677994 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:15.677953 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:20.677703 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:20.677662 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:25.677812 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:25.677763 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:08:28.847376 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.847351 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 19:08:28.889566 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.889525 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle\") pod \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " Jul 01 19:08:28.889744 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.889585 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls\") pod \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\" (UID: \"ba5c9b1a-46ff-4794-985b-6e2a883239e0\") " Jul 01 19:08:28.889964 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.889939 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ba5c9b1a-46ff-4794-985b-6e2a883239e0" (UID: "ba5c9b1a-46ff-4794-985b-6e2a883239e0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:08:28.891630 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.891607 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ba5c9b1a-46ff-4794-985b-6e2a883239e0" (UID: "ba5c9b1a-46ff-4794-985b-6e2a883239e0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:08:28.990417 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.990325 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba5c9b1a-46ff-4794-985b-6e2a883239e0-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:08:28.990417 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.990363 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba5c9b1a-46ff-4794-985b-6e2a883239e0-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:08:28.991310 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.991284 2554 generic.go:356] "Generic (PLEG): container finished" podID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerID="7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07" exitCode=0 Jul 01 19:08:28.991407 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.991337 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" Jul 01 19:08:28.991407 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.991371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" event={"ID":"ba5c9b1a-46ff-4794-985b-6e2a883239e0","Type":"ContainerDied","Data":"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07"} Jul 01 19:08:28.991502 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.991423 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4" event={"ID":"ba5c9b1a-46ff-4794-985b-6e2a883239e0","Type":"ContainerDied","Data":"6e11e03e1958e23614a6cb965510d9aef49dfc9dd5f07a9142d7e265be0ac267"} Jul 01 19:08:28.991502 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.991446 2554 scope.go:117] "RemoveContainer" containerID="7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07" Jul 01 19:08:28.999955 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:28.999939 2554 scope.go:117] "RemoveContainer" containerID="7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07" Jul 01 19:08:29.000225 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:08:29.000209 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07\": container with ID starting with 7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07 not found: ID does not exist" containerID="7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07" Jul 01 19:08:29.000269 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:29.000232 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07"} err="failed to get container status \"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07\": rpc error: code = NotFound desc = could not find container \"7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07\": container with ID starting with 7dafdcedf66327b13ef48182e008626ff8c3e586d2c77936be7700a96af79f07 not found: ID does not exist" Jul 01 19:08:29.012264 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:29.012239 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 19:08:29.018265 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:29.018237 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-af621-7d4fd998f4-sv5l4"] Jul 01 19:08:29.824528 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:29.824497 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" path="/var/lib/kubelet/pods/ba5c9b1a-46ff-4794-985b-6e2a883239e0/volumes" Jul 01 19:08:58.946161 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.946085 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:08:58.946587 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.946470 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" Jul 01 19:08:58.946587 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.946483 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" Jul 01 19:08:58.946587 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.946539 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5c9b1a-46ff-4794-985b-6e2a883239e0" containerName="ensemble-graph-af621" Jul 01 19:08:58.949470 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.949440 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:58.952108 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.952084 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-1bf2c-kube-rbac-proxy-sar-config\"" Jul 01 19:08:58.952221 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.952106 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-1bf2c-serving-cert\"" Jul 01 19:08:58.960091 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:58.960068 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:08:59.050867 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.050832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.051032 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.050886 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.152048 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.152016 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.152215 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.152065 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.152756 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.152738 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.154436 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.154420 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls\") pod \"splitter-graph-1bf2c-997f87564-8nkrj\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.260388 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.260360 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:08:59.393652 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:08:59.393478 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:08:59.396096 ip-10-0-139-178 kubenswrapper[2554]: W0701 19:08:59.396069 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79262580_0764_4917_876d_6523169babe7.slice/crio-17795d1eef67b2319bc16f78ea7fdda39a53bd5d889d37dea14cb44b6789c695 WatchSource:0}: Error finding container 17795d1eef67b2319bc16f78ea7fdda39a53bd5d889d37dea14cb44b6789c695: Status 404 returned error can't find the container with id 17795d1eef67b2319bc16f78ea7fdda39a53bd5d889d37dea14cb44b6789c695 Jul 01 19:09:00.138121 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:00.138088 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" event={"ID":"79262580-0764-4917-876d-6523169babe7","Type":"ContainerStarted","Data":"d3ac67856de60df5e01ab84b8441f282ddfb3b54da438c751f1c8c11255b5d7a"} Jul 01 19:09:00.138580 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:00.138140 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" event={"ID":"79262580-0764-4917-876d-6523169babe7","Type":"ContainerStarted","Data":"17795d1eef67b2319bc16f78ea7fdda39a53bd5d889d37dea14cb44b6789c695"} Jul 01 19:09:00.138580 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:00.138176 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:09:00.155362 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:00.155309 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podStartSLOduration=2.1552925370000002 podStartE2EDuration="2.155292537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:09:00.153718428 +0000 UTC m=+1898.756878335" watchObservedRunningTime="2026-07-01 19:09:00.155292537 +0000 UTC m=+1898.758452430" Jul 01 19:09:06.148670 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:06.148638 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:09:08.483084 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:08.483046 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:09:08.483525 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:08.483224 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" containerID="cri-o://7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17" gracePeriod=30 Jul 01 19:09:09.006586 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:09.006554 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:09:09.006756 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:09.006736 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" containerID="cri-o://d3ac67856de60df5e01ab84b8441f282ddfb3b54da438c751f1c8c11255b5d7a" gracePeriod=30 Jul 01 19:09:10.989000 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:10.988962 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:11.147303 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:11.147265 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:15.988766 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:15.988728 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:16.146868 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:16.146832 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:20.988641 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:20.988601 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:20.989037 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:20.988719 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:09:21.146463 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:21.146414 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:21.146641 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:21.146544 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:09:25.988979 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:25.988933 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:26.146503 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:26.146439 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:30.988396 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:30.988357 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:31.146774 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:31.146739 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:35.988736 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:35.988697 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:36.146935 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:36.146901 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:09:38.658326 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.658304 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:09:38.790973 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.790939 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle\") pod \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " Jul 01 19:09:38.791153 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.791048 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls\") pod \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\" (UID: \"12a0c79a-b5f5-43ef-a3db-b82cca330ffa\") " Jul 01 19:09:38.791228 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.791192 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "12a0c79a-b5f5-43ef-a3db-b82cca330ffa" (UID: "12a0c79a-b5f5-43ef-a3db-b82cca330ffa"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:09:38.791288 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.791251 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:09:38.793063 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.793041 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "12a0c79a-b5f5-43ef-a3db-b82cca330ffa" (UID: "12a0c79a-b5f5-43ef-a3db-b82cca330ffa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:09:38.891850 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:38.891814 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12a0c79a-b5f5-43ef-a3db-b82cca330ffa-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:09:39.307496 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.307466 2554 generic.go:356] "Generic (PLEG): container finished" podID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerID="7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17" exitCode=0 Jul 01 19:09:39.307673 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.307503 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" Jul 01 19:09:39.307673 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.307557 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" event={"ID":"12a0c79a-b5f5-43ef-a3db-b82cca330ffa","Type":"ContainerDied","Data":"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17"} Jul 01 19:09:39.307673 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.307611 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b" event={"ID":"12a0c79a-b5f5-43ef-a3db-b82cca330ffa","Type":"ContainerDied","Data":"039d8615fb352089bf59b5f16b5a1d1c6aa15a00b3f8a625dab4dc2640899d00"} Jul 01 19:09:39.307673 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.307628 2554 scope.go:117] "RemoveContainer" containerID="7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17" Jul 01 19:09:39.309024 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.309004 2554 generic.go:356] "Generic (PLEG): container finished" podID="79262580-0764-4917-876d-6523169babe7" containerID="d3ac67856de60df5e01ab84b8441f282ddfb3b54da438c751f1c8c11255b5d7a" exitCode=137 Jul 01 19:09:39.309125 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.309031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" event={"ID":"79262580-0764-4917-876d-6523169babe7","Type":"ContainerDied","Data":"d3ac67856de60df5e01ab84b8441f282ddfb3b54da438c751f1c8c11255b5d7a"} Jul 01 19:09:39.316171 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.316158 2554 scope.go:117] "RemoveContainer" containerID="7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17" Jul 01 19:09:39.316405 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:09:39.316386 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17\": container with ID starting with 7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17 not found: ID does not exist" containerID="7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17" Jul 01 19:09:39.316466 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.316411 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17"} err="failed to get container status \"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17\": rpc error: code = NotFound desc = could not find container \"7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17\": container with ID starting with 7decc2258de1a05fd5405347a088ecb867da9203200857bf766f75890fab4d17 not found: ID does not exist" Jul 01 19:09:39.339154 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.339123 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:09:39.341114 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.341095 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-c3262-557db7f5b8-g279b"] Jul 01 19:09:39.654512 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.654490 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:09:39.797713 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.797682 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls\") pod \"79262580-0764-4917-876d-6523169babe7\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " Jul 01 19:09:39.798076 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.797775 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle\") pod \"79262580-0764-4917-876d-6523169babe7\" (UID: \"79262580-0764-4917-876d-6523169babe7\") " Jul 01 19:09:39.798150 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.798129 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "79262580-0764-4917-876d-6523169babe7" (UID: "79262580-0764-4917-876d-6523169babe7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:09:39.799806 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.799781 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "79262580-0764-4917-876d-6523169babe7" (UID: "79262580-0764-4917-876d-6523169babe7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:09:39.824328 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.824262 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" path="/var/lib/kubelet/pods/12a0c79a-b5f5-43ef-a3db-b82cca330ffa/volumes" Jul 01 19:09:39.899334 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.899288 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79262580-0764-4917-876d-6523169babe7-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:09:39.899334 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:39.899330 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79262580-0764-4917-876d-6523169babe7-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:09:40.314762 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:40.314739 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" Jul 01 19:09:40.314930 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:40.314786 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj" event={"ID":"79262580-0764-4917-876d-6523169babe7","Type":"ContainerDied","Data":"17795d1eef67b2319bc16f78ea7fdda39a53bd5d889d37dea14cb44b6789c695"} Jul 01 19:09:40.314930 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:40.314861 2554 scope.go:117] "RemoveContainer" containerID="d3ac67856de60df5e01ab84b8441f282ddfb3b54da438c751f1c8c11255b5d7a" Jul 01 19:09:40.332360 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:40.332337 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:09:40.336775 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:40.336757 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1bf2c-997f87564-8nkrj"] Jul 01 19:09:41.824396 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:09:41.824365 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79262580-0764-4917-876d-6523169babe7" path="/var/lib/kubelet/pods/79262580-0764-4917-876d-6523169babe7/volumes" Jul 01 19:10:09.235681 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.235651 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:10:09.236059 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.235984 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" Jul 01 19:10:09.236059 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.235995 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" Jul 01 19:10:09.236059 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.236003 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" Jul 01 19:10:09.236059 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.236009 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" Jul 01 19:10:09.236192 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.236071 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="79262580-0764-4917-876d-6523169babe7" containerName="splitter-graph-1bf2c" Jul 01 19:10:09.236192 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.236078 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="12a0c79a-b5f5-43ef-a3db-b82cca330ffa" containerName="sequence-graph-c3262" Jul 01 19:10:09.238860 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.238844 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.241089 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.241067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4f643-serving-cert\"" Jul 01 19:10:09.241184 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.241127 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-qplkr\"" Jul 01 19:10:09.241184 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.241155 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 19:10:09.241819 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.241803 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4f643-kube-rbac-proxy-sar-config\"" Jul 01 19:10:09.247363 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.247344 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:10:09.257500 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.257472 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.257609 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.257530 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.358559 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.358519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.358726 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.358606 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.358766 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:10:09.358742 2554 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-4f643-serving-cert: secret "splitter-graph-4f643-serving-cert" not found Jul 01 19:10:09.358825 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:10:09.358815 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls podName:aa90ba40-136c-45ea-88c2-b1b84275bae4 nodeName:}" failed. No retries permitted until 2026-07-01 19:10:09.858797843 +0000 UTC m=+1968.461957714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls") pod "splitter-graph-4f643-59d4664c7d-tk7t6" (UID: "aa90ba40-136c-45ea-88c2-b1b84275bae4") : secret "splitter-graph-4f643-serving-cert" not found Jul 01 19:10:09.359187 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.359170 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.863088 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.863052 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:09.865441 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:09.865418 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") pod \"splitter-graph-4f643-59d4664c7d-tk7t6\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:10.150428 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.150338 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:10.280915 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.280892 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:10:10.282879 ip-10-0-139-178 kubenswrapper[2554]: W0701 19:10:10.282855 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa90ba40_136c_45ea_88c2_b1b84275bae4.slice/crio-453e759644fcae3ecfbe46e01d3b094bc76cba36de638e717da499e6ef21001a WatchSource:0}: Error finding container 453e759644fcae3ecfbe46e01d3b094bc76cba36de638e717da499e6ef21001a: Status 404 returned error can't find the container with id 453e759644fcae3ecfbe46e01d3b094bc76cba36de638e717da499e6ef21001a Jul 01 19:10:10.452642 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.452551 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" event={"ID":"aa90ba40-136c-45ea-88c2-b1b84275bae4","Type":"ContainerStarted","Data":"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1"} Jul 01 19:10:10.452642 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.452602 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" event={"ID":"aa90ba40-136c-45ea-88c2-b1b84275bae4","Type":"ContainerStarted","Data":"453e759644fcae3ecfbe46e01d3b094bc76cba36de638e717da499e6ef21001a"} Jul 01 19:10:10.452819 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.452670 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:10.470711 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:10.470668 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podStartSLOduration=1.4706540750000001 podStartE2EDuration="1.470654075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:10:10.469845478 +0000 UTC m=+1969.073005373" watchObservedRunningTime="2026-07-01 19:10:10.470654075 +0000 UTC m=+1969.073813988" Jul 01 19:10:16.463162 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:16.463079 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:10:18.716063 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.716027 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:10:18.719483 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.719446 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:18.721546 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.721515 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-43223-serving-cert\"" Jul 01 19:10:18.721546 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.721533 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-43223-kube-rbac-proxy-sar-config\"" Jul 01 19:10:18.726941 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.726921 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:10:18.836667 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.836630 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:18.836667 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.836673 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:18.938096 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.938058 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:18.938297 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.938109 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:18.938297 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:10:18.938226 2554 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-43223-serving-cert: secret "switch-graph-43223-serving-cert" not found Jul 01 19:10:18.938297 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:10:18.938298 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls podName:405c3fc8-43e9-40cb-bdab-a8a575984d31 nodeName:}" failed. No retries permitted until 2026-07-01 19:10:19.438277125 +0000 UTC m=+1978.041436998 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls") pod "switch-graph-43223-7d58669c44-z74rq" (UID: "405c3fc8-43e9-40cb-bdab-a8a575984d31") : secret "switch-graph-43223-serving-cert" not found Jul 01 19:10:18.938722 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:18.938702 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:19.442682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:19.442645 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:19.445114 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:19.445089 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") pod \"switch-graph-43223-7d58669c44-z74rq\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:19.630908 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:19.630876 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:19.764390 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:19.764365 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:10:19.766409 ip-10-0-139-178 kubenswrapper[2554]: W0701 19:10:19.766383 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405c3fc8_43e9_40cb_bdab_a8a575984d31.slice/crio-e565424566102ebd61a8e1cefc51bf047a260283241991daade03dba54c1fb63 WatchSource:0}: Error finding container e565424566102ebd61a8e1cefc51bf047a260283241991daade03dba54c1fb63: Status 404 returned error can't find the container with id e565424566102ebd61a8e1cefc51bf047a260283241991daade03dba54c1fb63 Jul 01 19:10:20.495958 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:20.495921 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" event={"ID":"405c3fc8-43e9-40cb-bdab-a8a575984d31","Type":"ContainerStarted","Data":"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940"} Jul 01 19:10:20.496112 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:20.496002 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" event={"ID":"405c3fc8-43e9-40cb-bdab-a8a575984d31","Type":"ContainerStarted","Data":"e565424566102ebd61a8e1cefc51bf047a260283241991daade03dba54c1fb63"} Jul 01 19:10:20.496112 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:20.496039 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:10:20.512802 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:20.512752 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podStartSLOduration=2.512735023 podStartE2EDuration="2.512735023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 19:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:10:20.5127205 +0000 UTC m=+1979.115880389" watchObservedRunningTime="2026-07-01 19:10:20.512735023 +0000 UTC m=+1979.115894920" Jul 01 19:10:26.505998 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:10:26.505973 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:12:21.929905 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:12:21.929877 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:12:21.934181 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:12:21.934162 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:17:21.955068 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:17:21.954962 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:17:21.960664 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:17:21.960647 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:18:23.938658 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:23.938623 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:18:23.939219 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:23.938844 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" containerID="cri-o://d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1" gracePeriod=30 Jul 01 19:18:26.460682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:26.460642 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:31.460790 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:31.460745 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:36.461138 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:36.461092 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:36.461579 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:36.461202 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:18:41.460626 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:41.460590 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:46.461087 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:46.461048 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:51.460716 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:51.460677 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:18:54.093412 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.093385 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:18:54.199312 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.199226 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") pod \"aa90ba40-136c-45ea-88c2-b1b84275bae4\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " Jul 01 19:18:54.199490 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.199358 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle\") pod \"aa90ba40-136c-45ea-88c2-b1b84275bae4\" (UID: \"aa90ba40-136c-45ea-88c2-b1b84275bae4\") " Jul 01 19:18:54.199790 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.199762 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "aa90ba40-136c-45ea-88c2-b1b84275bae4" (UID: "aa90ba40-136c-45ea-88c2-b1b84275bae4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:18:54.201534 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.201514 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aa90ba40-136c-45ea-88c2-b1b84275bae4" (UID: "aa90ba40-136c-45ea-88c2-b1b84275bae4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:18:54.300134 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.300102 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa90ba40-136c-45ea-88c2-b1b84275bae4-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:18:54.300134 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.300130 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa90ba40-136c-45ea-88c2-b1b84275bae4-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:18:54.743104 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.743068 2554 generic.go:356] "Generic (PLEG): container finished" podID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerID="d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1" exitCode=0 Jul 01 19:18:54.743104 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.743108 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" Jul 01 19:18:54.743330 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.743115 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" event={"ID":"aa90ba40-136c-45ea-88c2-b1b84275bae4","Type":"ContainerDied","Data":"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1"} Jul 01 19:18:54.743330 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.743170 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6" event={"ID":"aa90ba40-136c-45ea-88c2-b1b84275bae4","Type":"ContainerDied","Data":"453e759644fcae3ecfbe46e01d3b094bc76cba36de638e717da499e6ef21001a"} Jul 01 19:18:54.743330 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.743195 2554 scope.go:117] "RemoveContainer" containerID="d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1" Jul 01 19:18:54.751861 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.751846 2554 scope.go:117] "RemoveContainer" containerID="d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1" Jul 01 19:18:54.752108 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:18:54.752089 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1\": container with ID starting with d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1 not found: ID does not exist" containerID="d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1" Jul 01 19:18:54.752153 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.752117 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1"} err="failed to get container status \"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1\": rpc error: code = NotFound desc = could not find container \"d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1\": container with ID starting with d009a87cc726559380475e93c123edc180ba9143c0356d6fda5e4a8a4a5a29e1 not found: ID does not exist" Jul 01 19:18:54.765639 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.765619 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:18:54.770365 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:54.770334 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f643-59d4664c7d-tk7t6"] Jul 01 19:18:55.824678 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:18:55.824651 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" path="/var/lib/kubelet/pods/aa90ba40-136c-45ea-88c2-b1b84275bae4/volumes" Jul 01 19:22:21.979256 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:22:21.979144 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:22:21.986126 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:22:21.986106 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:26:38.057959 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:38.057925 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:26:38.058502 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:38.058132 2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" containerID="cri-o://49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940" gracePeriod=30 Jul 01 19:26:41.503729 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:41.503691 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:26:46.503704 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:46.503624 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:26:51.503996 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:51.503957 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:26:51.504392 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:51.504104 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:26:54.982302 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:54.982266 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:55.823876 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:55.823845 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:56.504088 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:56.504055 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:26:56.663179 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:56.663149 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:57.475409 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:57.475378 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:58.288539 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:58.288510 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:59.154706 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:59.154678 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:26:59.973397 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:26:59.973365 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:00.787819 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:00.787790 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:01.504231 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:01.504191 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:27:01.620611 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:01.620579 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:02.470159 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:02.470132 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:03.297614 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:03.297586 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:04.104293 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:04.104265 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:04.923813 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:04.923784 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:05.757853 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:05.757820 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-43223-7d58669c44-z74rq_405c3fc8-43e9-40cb-bdab-a8a575984d31/switch-graph-43223/0.log" Jul 01 19:27:06.503884 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:06.503843 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 19:27:08.204137 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.204114 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:27:08.216180 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.216161 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle\") pod \"405c3fc8-43e9-40cb-bdab-a8a575984d31\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " Jul 01 19:27:08.216294 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.216234 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") pod \"405c3fc8-43e9-40cb-bdab-a8a575984d31\" (UID: \"405c3fc8-43e9-40cb-bdab-a8a575984d31\") " Jul 01 19:27:08.216535 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.216511 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "405c3fc8-43e9-40cb-bdab-a8a575984d31" (UID: "405c3fc8-43e9-40cb-bdab-a8a575984d31"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 19:27:08.218112 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.218093 2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "405c3fc8-43e9-40cb-bdab-a8a575984d31" (UID: "405c3fc8-43e9-40cb-bdab-a8a575984d31"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 19:27:08.317401 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.317328 2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/405c3fc8-43e9-40cb-bdab-a8a575984d31-openshift-service-ca-bundle\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:27:08.317401 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.317354 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/405c3fc8-43e9-40cb-bdab-a8a575984d31-proxy-tls\") on node \"ip-10-0-139-178.ec2.internal\" DevicePath \"\"" Jul 01 19:27:08.877648 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.877613 2554 generic.go:356] "Generic (PLEG): container finished" podID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerID="49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940" exitCode=0 Jul 01 19:27:08.877842 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.877656 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" Jul 01 19:27:08.877842 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.877699 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" event={"ID":"405c3fc8-43e9-40cb-bdab-a8a575984d31","Type":"ContainerDied","Data":"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940"} Jul 01 19:27:08.877842 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.877756 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq" event={"ID":"405c3fc8-43e9-40cb-bdab-a8a575984d31","Type":"ContainerDied","Data":"e565424566102ebd61a8e1cefc51bf047a260283241991daade03dba54c1fb63"} Jul 01 19:27:08.877842 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.877785 2554 scope.go:117] "RemoveContainer" containerID="49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940" Jul 01 19:27:08.888198 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.888182 2554 scope.go:117] "RemoveContainer" containerID="49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940" Jul 01 19:27:08.888508 ip-10-0-139-178 kubenswrapper[2554]: E0701 19:27:08.888483 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940\": container with ID starting with 49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940 not found: ID does not exist" containerID="49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940" Jul 01 19:27:08.888587 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.888519 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940"} err="failed to get container status \"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940\": rpc error: code = NotFound desc = could not find container \"49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940\": container with ID starting with 49cfa4acd38836a3e9a664c9a7f85c86fa72cc0946925a574b5ccbfb75479940 not found: ID does not exist" Jul 01 19:27:08.905387 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.905368 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:27:08.915682 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:08.915666 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-43223-7d58669c44-z74rq"] Jul 01 19:27:09.825382 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:09.825338 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" path="/var/lib/kubelet/pods/405c3fc8-43e9-40cb-bdab-a8a575984d31/volumes" Jul 01 19:27:12.667936 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:12.667904 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jkrhv_7bae8d16-a7ec-416b-874a-385f16b4b780/global-pull-secret-syncer/0.log" Jul 01 19:27:12.791355 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:12.791324 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hkn6f_e6ea3845-6f87-4d9d-b346-d33388480adb/konnectivity-agent/0.log" Jul 01 19:27:12.901596 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:12.901569 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-178.ec2.internal_91f221773be377b95c268f7c48aa904d/haproxy/0.log" Jul 01 19:27:17.464040 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.464010 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-n5g6z_ae045f82-c67a-48d1-9731-f3f999363168/cluster-monitoring-operator/0.log" Jul 01 19:27:17.608237 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.608211 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75d5fd74f6-wjtks_1499c9fb-badd-41de-ae9d-1a1df46fd6b6/metrics-server/0.log" Jul 01 19:27:17.641815 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.641791 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-hb9zn_cea34acc-285a-4e38-9437-9b44f884ab4b/monitoring-plugin/0.log" Jul 01 19:27:17.679226 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.679207 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wqvl_7715c2a2-d1e2-4110-b586-93c88f863638/node-exporter/0.log" Jul 01 19:27:17.710195 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.710172 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wqvl_7715c2a2-d1e2-4110-b586-93c88f863638/kube-rbac-proxy/0.log" Jul 01 19:27:17.733755 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:17.733703 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6wqvl_7715c2a2-d1e2-4110-b586-93c88f863638/init-textfile/0.log" Jul 01 19:27:18.246157 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.246130 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-f6xlc_bab11438-9d3e-4878-8984-47e11923a323/prometheus-operator/0.log" Jul 01 19:27:18.271642 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.271620 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-f6xlc_bab11438-9d3e-4878-8984-47e11923a323/kube-rbac-proxy/0.log" Jul 01 19:27:18.298956 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.298936 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-rb6vj_edf1f520-483f-4347-a2ee-1fb9706c9138/prometheus-operator-admission-webhook/0.log" Jul 01 19:27:18.332741 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.332724 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-695b6c96f5-sgjmq_54b9a7ca-e2de-444a-a4cf-606301dcb6ec/telemeter-client/0.log" Jul 01 19:27:18.369182 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.369166 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-695b6c96f5-sgjmq_54b9a7ca-e2de-444a-a4cf-606301dcb6ec/reload/0.log" Jul 01 19:27:18.401796 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.401778 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-695b6c96f5-sgjmq_54b9a7ca-e2de-444a-a4cf-606301dcb6ec/kube-rbac-proxy/0.log" Jul 01 19:27:18.440559 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.440539 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/thanos-query/0.log" Jul 01 19:27:18.476586 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.476565 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/kube-rbac-proxy-web/0.log" Jul 01 19:27:18.504914 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.504864 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/kube-rbac-proxy/0.log" Jul 01 19:27:18.533372 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.533353 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/prom-label-proxy/0.log" Jul 01 19:27:18.568938 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.568921 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/kube-rbac-proxy-rules/0.log" Jul 01 19:27:18.607953 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:18.607934 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fcd97b69-86s5d_f1c3e5b9-6cad-4841-8582-b984e8f25847/kube-rbac-proxy-metrics/0.log" Jul 01 19:27:20.832597 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:20.832571 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-68ffd9f4d6-w7x85_e40a8f1e-2021-4625-8fe4-8186458b1318/console/0.log" Jul 01 19:27:21.150495 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150398 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2"] Jul 01 19:27:21.150864 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150846 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" Jul 01 19:27:21.150864 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150864 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" Jul 01 19:27:21.150984 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150880 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" Jul 01 19:27:21.150984 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150886 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" Jul 01 19:27:21.150984 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150949 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="405c3fc8-43e9-40cb-bdab-a8a575984d31" containerName="switch-graph-43223" Jul 01 19:27:21.150984 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.150961 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa90ba40-136c-45ea-88c2-b1b84275bae4" containerName="splitter-graph-4f643" Jul 01 19:27:21.155357 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.155338 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.157809 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.157791 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gprgv\"/\"kube-root-ca.crt\"" Jul 01 19:27:21.158563 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.158546 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gprgv\"/\"default-dockercfg-9l27g\"" Jul 01 19:27:21.158615 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.158572 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gprgv\"/\"openshift-service-ca.crt\"" Jul 01 19:27:21.161059 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.161041 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2"] Jul 01 19:27:21.220986 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.220961 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-podres\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.221113 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.220993 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzv4m\" (UniqueName: \"kubernetes.io/projected/4462cac5-a35a-4a15-ae38-6255d20cb228-kube-api-access-dzv4m\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.221113 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.221080 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-sys\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.221113 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.221108 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-lib-modules\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.221245 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.221143 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-proc\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322397 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-podres\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322546 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322408 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzv4m\" (UniqueName: \"kubernetes.io/projected/4462cac5-a35a-4a15-ae38-6255d20cb228-kube-api-access-dzv4m\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322546 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322500 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-sys\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322546 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322527 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-lib-modules\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322710 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322544 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-podres\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322710 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322570 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-proc\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322710 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322590 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-sys\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322710 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322659 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-proc\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.322710 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.322679 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4462cac5-a35a-4a15-ae38-6255d20cb228-lib-modules\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.330930 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.330914 2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzv4m\" (UniqueName: \"kubernetes.io/projected/4462cac5-a35a-4a15-ae38-6255d20cb228-kube-api-access-dzv4m\") pod \"perf-node-gather-daemonset-fcfp2\" (UID: \"4462cac5-a35a-4a15-ae38-6255d20cb228\") " pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.465864 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.465807 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.593942 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.593920 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2"] Jul 01 19:27:21.595986 ip-10-0-139-178 kubenswrapper[2554]: W0701 19:27:21.595961 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4462cac5_a35a_4a15_ae38_6255d20cb228.slice/crio-78d91d90613ae1826fdb62581bd087eeee61dc1b16cb0b649ee27cb3862a1865 WatchSource:0}: Error finding container 78d91d90613ae1826fdb62581bd087eeee61dc1b16cb0b649ee27cb3862a1865: Status 404 returned error can't find the container with id 78d91d90613ae1826fdb62581bd087eeee61dc1b16cb0b649ee27cb3862a1865 Jul 01 19:27:21.934289 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.934255 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" event={"ID":"4462cac5-a35a-4a15-ae38-6255d20cb228","Type":"ContainerStarted","Data":"e553680a324d8979e201b31ac1c4be2ee0f5ee565a1ca0414aa6cba33f8a55a7"} Jul 01 19:27:21.934796 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.934317 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" event={"ID":"4462cac5-a35a-4a15-ae38-6255d20cb228","Type":"ContainerStarted","Data":"78d91d90613ae1826fdb62581bd087eeee61dc1b16cb0b649ee27cb3862a1865"} Jul 01 19:27:21.934796 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.934373 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:21.951611 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:21.951563 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" podStartSLOduration=0.951550121 podStartE2EDuration="951.550121ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 19:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 19:27:21.950135503 +0000 UTC m=+3000.553295396" watchObservedRunningTime="2026-07-01 19:27:21.951550121 +0000 UTC m=+3000.554710015" Jul 01 19:27:22.004795 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.004768 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:27:22.012438 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.012421 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:27:22.438674 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.438643 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zr9jd_fce2c7b4-6610-44c0-882d-d816c734aa6e/dns/0.log" Jul 01 19:27:22.458447 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.458423 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zr9jd_fce2c7b4-6610-44c0-882d-d816c734aa6e/kube-rbac-proxy/0.log" Jul 01 19:27:22.529112 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.529085 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nb24j_1d5ef2ef-3ddb-46b8-8f60-aaad32410974/dns-node-resolver/0.log" Jul 01 19:27:22.988521 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:22.988490 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-d65c54b89-nfd6t_f955947e-1e6b-4b74-936f-adb2d918f9f3/registry/0.log" Jul 01 19:27:23.011028 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:23.010995 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5f79x_cf08c72b-7225-4ad1-9989-82b43ce8337a/node-ca/0.log" Jul 01 19:27:24.251464 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:24.251434 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b72ng_e455c3a6-5452-4971-8b72-072592a96d92/serve-healthcheck-canary/0.log" Jul 01 19:27:24.686388 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:24.686280 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ljtr7_d3ac98d7-225d-42ff-bd2b-4ad4f43e3090/kube-rbac-proxy/0.log" Jul 01 19:27:24.708508 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:24.708476 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ljtr7_d3ac98d7-225d-42ff-bd2b-4ad4f43e3090/exporter/0.log" Jul 01 19:27:24.733596 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:24.733574 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ljtr7_d3ac98d7-225d-42ff-bd2b-4ad4f43e3090/extractor/0.log" Jul 01 19:27:27.065364 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:27.065337 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-67d95fdc48-jpg24_c857f9d9-6d3b-4206-8c19-a3021aac240f/manager/0.log" Jul 01 19:27:27.186423 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:27.186397 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-84b54b7f97-6wklr_27b5a3fc-731a-4fba-aba1-326005c49c6b/manager/0.log" Jul 01 19:27:27.948235 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:27.948205 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gprgv/perf-node-gather-daemonset-fcfp2" Jul 01 19:27:32.177857 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:32.177826 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-m55vr_87f2696b-218e-4902-8a03-71a1c19b90c7/kube-storage-version-migrator-operator/1.log" Jul 01 19:27:32.178514 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:32.178497 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-m55vr_87f2696b-218e-4902-8a03-71a1c19b90c7/kube-storage-version-migrator-operator/0.log" Jul 01 19:27:33.362128 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.362101 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/kube-multus-additional-cni-plugins/0.log" Jul 01 19:27:33.386805 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.386785 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/egress-router-binary-copy/0.log" Jul 01 19:27:33.413053 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.413034 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/cni-plugins/0.log" Jul 01 19:27:33.443647 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.443631 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/bond-cni-plugin/0.log" Jul 01 19:27:33.491082 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.491064 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/routeoverride-cni/0.log" Jul 01 19:27:33.527772 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.527751 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/whereabouts-cni-bincopy/0.log" Jul 01 19:27:33.558785 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.558759 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pq84p_1a6078ce-b69f-40ab-b65a-c662af43cffe/whereabouts-cni/0.log" Jul 01 19:27:33.787649 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.787625 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-szl4z_fb7c0baf-693d-4bc2-be0d-0346903c7574/kube-multus/0.log" Jul 01 19:27:33.809004 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.808987 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6hvdk_2c9cac5a-22da-46f2-a514-049a7075ea4a/network-metrics-daemon/0.log" Jul 01 19:27:33.831897 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:33.831881 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6hvdk_2c9cac5a-22da-46f2-a514-049a7075ea4a/kube-rbac-proxy/0.log" Jul 01 19:27:34.693833 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.693802 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-controller/0.log" Jul 01 19:27:34.713885 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.713862 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/0.log" Jul 01 19:27:34.714071 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.713904 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovn-acl-logging/1.log" Jul 01 19:27:34.733762 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.733722 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/kube-rbac-proxy-node/0.log" Jul 01 19:27:34.758869 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.758848 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 19:27:34.779947 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.779923 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/northd/0.log" Jul 01 19:27:34.801425 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.801404 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/nbdb/0.log" Jul 01 19:27:34.827844 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.827823 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/sbdb/0.log" Jul 01 19:27:34.877347 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:34.877324 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2nnxm_cb13e17c-1220-4060-9d8c-4d2e7bdea571/ovnkube-controller/0.log" Jul 01 19:27:36.335049 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:36.335020 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-45t9c_8983e4a6-bc80-4093-91b1-04f0f95be3c2/check-endpoints/0.log" Jul 01 19:27:36.409980 ip-10-0-139-178 kubenswrapper[2554]: I0701 19:27:36.409957 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-twz9h_1bdf0907-9f42-406f-a37e-49e5cc80e144/network-check-target-container/0.log"