Jul 21 08:16:49.571056 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 21 08:16:49.571067 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 21 08:16:49.571077 ip-10-0-142-97 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 21 08:16:49.571388 ip-10-0-142-97 systemd[1]: Failed to start Kubernetes Kubelet. Jul 21 08:16:59.736698 ip-10-0-142-97 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 21 08:16:59.736722 ip-10-0-142-97 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 97c380d1e9d2427988f171eb9f30410f -- Jul 21 08:18:56.120665 ip-10-0-142-97 systemd[1]: Starting Kubernetes Kubelet... Jul 21 08:18:56.518181 ip-10-0-142-97 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 08:18:56.518181 ip-10-0-142-97 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 08:18:56.518181 ip-10-0-142-97 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 08:18:56.518181 ip-10-0-142-97 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 08:18:56.518181 ip-10-0-142-97 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 08:18:56.518903 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.518256 2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 08:18:56.524172 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524150 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.524172 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524171 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524176 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524179 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524183 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524186 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524189 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524192 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524194 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524197 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524200 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524202 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524205 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524208 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524210 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524213 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524217 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524220 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524222 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524224 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524227 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524229 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.524247 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524232 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524234 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524236 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524239 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524242 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524245 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524247 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524250 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524252 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524255 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524258 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524261 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524263 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524285 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524288 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524290 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524292 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524295 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524298 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.524757 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524300 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524302 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524305 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524307 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524311 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524314 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524316 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524318 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524322 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524324 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524327 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524329 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524332 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524334 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524337 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524340 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524343 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524346 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524348 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524351 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.525212 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524353 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524358 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524362 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524365 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524368 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524371 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524373 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524376 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524379 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524381 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524383 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524386 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524388 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524391 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524393 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524395 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524398 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524402 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524405 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.525711 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524408 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524410 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524413 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524416 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524418 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524421 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524424 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524427 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524431 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524433 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524436 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524438 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524441 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524444 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524447 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524452 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524455 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524458 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.524461 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.526180 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525320 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525328 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525332 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525335 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525338 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525341 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525345 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525348 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525350 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525353 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525356 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525358 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525361 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525363 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525366 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525368 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525377 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525380 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525383 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525385 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.526694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525388 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525391 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525393 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525396 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525398 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525401 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525403 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525406 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525409 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525411 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525414 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525416 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525418 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525421 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525424 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525426 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525429 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525431 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525433 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.527203 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525436 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525438 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525441 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525443 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525446 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525448 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525451 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525454 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525457 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525459 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525469 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525472 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525474 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525477 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525480 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525482 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525485 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525487 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525490 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525492 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.527692 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525495 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525497 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525499 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525502 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525504 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525506 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525509 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525511 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525515 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525519 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525521 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525524 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525528 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525530 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525533 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525535 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525537 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525540 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525543 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.528169 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525546 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525548 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525552 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525556 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525570 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525572 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525575 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525578 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525580 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525583 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525585 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525587 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525590 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525592 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525595 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525597 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525601 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525604 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525607 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.528639 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525610 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.525612 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526068 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526074 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526079 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526082 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526085 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526088 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526091 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526093 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526096 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526099 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526102 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526106 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526109 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526112 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526114 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526125 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526127 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.529141 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526130 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526132 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526140 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526143 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526145 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526148 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526152 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526155 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526158 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526161 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526164 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526166 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526168 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526171 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526173 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526175 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526177 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526180 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526183 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526186 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.529623 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526188 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526191 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526193 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526196 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526198 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526201 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526203 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526206 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526208 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526211 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526214 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526222 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526225 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526227 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526230 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526233 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526236 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526238 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526241 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526243 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.530106 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526245 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526248 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526251 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526253 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526255 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526258 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526260 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526263 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526279 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526281 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526284 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526286 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526289 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526291 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526294 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526296 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526299 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526301 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526304 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526306 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.530605 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526308 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526311 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526314 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526316 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526326 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526329 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526332 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526334 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526336 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526339 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526341 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526344 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526346 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526348 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526354 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526357 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526359 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526361 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526364 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.531143 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526366 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526369 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.526371 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526459 2569 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526481 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526493 2569 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526498 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526502 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526505 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526509 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526513 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526516 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526523 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526526 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526529 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526533 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526536 2569 flags.go:64] FLAG: --cgroup-root="" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526539 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526547 2569 flags.go:64] FLAG: --client-ca-file="" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526550 2569 flags.go:64] FLAG: --cloud-provider="external" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526553 2569 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526559 2569 flags.go:64] FLAG: --cluster-domain="" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526562 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526565 2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 08:18:56.531614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526568 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526571 2569 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526575 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526580 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526584 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526586 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526590 2569 flags.go:64] FLAG: --contention-profiling="false" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526593 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526596 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526599 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526602 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526606 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526614 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526617 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526620 2569 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526623 2569 flags.go:64] FLAG: --enable-server="true" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526626 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526633 2569 flags.go:64] FLAG: --event-burst="100" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526636 2569 flags.go:64] FLAG: --event-qps="50" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526641 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526643 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526646 2569 flags.go:64] FLAG: --eviction-hard="" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526650 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526653 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526656 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 08:18:56.532191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526658 2569 flags.go:64] FLAG: --eviction-soft="" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526661 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526669 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526672 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526675 2569 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526678 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526681 2569 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526684 2569 flags.go:64] FLAG: --feature-gates="" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526689 2569 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526692 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526697 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526700 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526703 2569 flags.go:64] FLAG: --healthz-port="10248" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526706 2569 flags.go:64] FLAG: --help="false" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526709 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526712 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526714 2569 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526717 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526720 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526724 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526726 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526729 2569 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526732 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 08:18:56.532826 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526735 2569 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526737 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526740 2569 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526745 2569 flags.go:64] FLAG: --kube-reserved="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526748 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526750 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526753 2569 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526756 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526759 2569 flags.go:64] FLAG: --lock-file="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526761 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526764 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526767 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526779 2569 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526782 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526785 2569 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526787 2569 flags.go:64] FLAG: --logging-format="text" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526790 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526794 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526796 2569 flags.go:64] FLAG: --manifest-url="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526800 2569 flags.go:64] FLAG: --manifest-url-header="" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526804 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526807 2569 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526811 2569 flags.go:64] FLAG: --max-pods="110" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526814 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526817 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 08:18:56.533436 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526819 2569 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526822 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526825 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526828 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526831 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526838 2569 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526841 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526844 2569 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526847 2569 flags.go:64] FLAG: --pod-cidr="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526849 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526859 2569 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526863 2569 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526866 2569 flags.go:64] FLAG: --pods-per-core="0" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526869 2569 flags.go:64] FLAG: --port="10250" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526872 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526875 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b346f59cde36e4c" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526878 2569 flags.go:64] FLAG: --qos-reserved="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526881 2569 flags.go:64] FLAG: --read-only-port="10255" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526884 2569 flags.go:64] FLAG: --register-node="true" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526886 2569 flags.go:64] FLAG: --register-with-taints="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526895 2569 flags.go:64] FLAG: --registry-burst="10" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526898 2569 flags.go:64] FLAG: --registry-qps="5" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526901 2569 flags.go:64] FLAG: --reserved-cpus="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526904 2569 flags.go:64] FLAG: --reserved-memory="" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526908 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 08:18:56.534042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526911 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526914 2569 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526917 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526920 2569 flags.go:64] FLAG: --runonce="false" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526923 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526926 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526929 2569 flags.go:64] FLAG: --seccomp-default="false" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526932 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526935 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526938 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526941 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526944 2569 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526947 2569 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526949 2569 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526952 2569 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526955 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526958 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526961 2569 flags.go:64] FLAG: --system-cgroups="" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526966 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526971 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526974 2569 flags.go:64] FLAG: --tls-cert-file="" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526977 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526983 2569 flags.go:64] FLAG: --tls-min-version="" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526986 2569 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526989 2569 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 08:18:56.534699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526991 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526994 2569 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.526997 2569 flags.go:64] FLAG: --v="2" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.527006 2569 flags.go:64] FLAG: --version="false" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.527011 2569 flags.go:64] FLAG: --vmodule="" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.527015 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.527018 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527200 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527205 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527208 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527211 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527213 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527216 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527219 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527221 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527224 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527227 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527229 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527231 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527234 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527236 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.535615 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527239 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527241 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527243 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527246 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527250 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527252 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527255 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527257 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527260 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527262 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527277 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527280 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527282 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527285 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527288 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527299 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527303 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527305 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527307 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527310 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527313 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.536394 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527315 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527318 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527320 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527323 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527325 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527327 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527330 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527332 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527335 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527337 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527340 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527342 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527345 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527347 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527350 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527353 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527356 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527360 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527364 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.536936 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527367 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527369 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527372 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527374 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527376 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527379 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527381 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527384 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527392 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527394 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527397 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527402 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527406 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527409 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527411 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527414 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527417 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527419 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527422 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.537434 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527425 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527428 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527430 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527433 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527436 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527439 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527441 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527444 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527446 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527451 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527453 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527456 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527459 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527462 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527464 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527467 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527475 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527478 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527480 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527483 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.537897 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527485 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527488 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527495 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527498 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527503 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.527507 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.528262 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.537506 2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.537526 2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.538339 2569 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 08:18:56.538407 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.538348 2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538419 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538426 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538429 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538433 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538436 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538439 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538442 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538445 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538448 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538451 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538453 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538456 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538458 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538461 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538464 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538466 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538470 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538472 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538475 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.538667 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538477 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538480 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538482 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538485 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538487 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538491 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538495 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538498 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538501 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538504 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538507 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538509 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538512 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538515 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538519 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538522 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538524 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538527 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538530 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.539151 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538533 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538535 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538538 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538541 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538543 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538546 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538548 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538551 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538555 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538557 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538560 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538562 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538565 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538568 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538570 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538573 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538575 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538578 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538581 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538584 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.539637 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538586 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538589 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538591 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538594 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538596 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538599 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538602 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538605 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538607 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538610 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538612 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538614 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538618 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538622 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538624 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538627 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538630 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538633 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538635 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.540136 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538638 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538641 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538644 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538647 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538650 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538652 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538655 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538657 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538659 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538662 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538664 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538667 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538670 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538672 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538675 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538678 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538681 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538683 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538686 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.540626 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538689 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538693 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538696 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.538702 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538826 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538832 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538835 2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538838 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538841 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538843 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538846 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538849 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538851 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538854 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538856 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 08:18:56.541090 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538859 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538862 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538865 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538868 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538870 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538873 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538876 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538878 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538881 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538883 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538887 2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538892 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538895 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538897 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538900 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538902 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538905 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538910 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538914 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 08:18:56.541592 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538917 2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538919 2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538922 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538924 2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538927 2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538929 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538932 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538934 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538937 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538939 2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538942 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538944 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538947 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538949 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538951 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538954 2569 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538957 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538959 2569 feature_gate.go:328] unrecognized feature gate: Example Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538962 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538965 2569 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 08:18:56.542064 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538967 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538970 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538972 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538975 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538977 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538980 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538983 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538985 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538987 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538990 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538992 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538995 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.538997 2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539000 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539002 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539005 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539007 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539009 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539011 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539014 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 08:18:56.542573 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539017 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539019 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539021 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539024 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539026 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539028 2569 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539031 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539033 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539037 2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539039 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539042 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539044 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539046 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539049 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539051 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539054 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539057 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539060 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539062 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539064 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 08:18:56.543060 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539067 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539069 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539072 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539074 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539077 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539079 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539083 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539086 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:56.539090 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.539095 2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.539243 2569 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 08:18:56.543581 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.542828 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 08:18:56.544280 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.544256 2569 server.go:1019] "Starting client certificate rotation" Jul 21 08:18:56.544387 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.544368 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 08:18:56.544430 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.544401 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 08:18:56.547014 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.546999 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 08:18:56.548359 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.548325 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 08:18:56.559908 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.559875 2569 log.go:25] "Validated CRI v1 runtime API" Jul 21 08:18:56.562229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.562211 2569 log.go:25] "Validated CRI v1 image API" Jul 21 08:18:56.563707 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.563691 2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 08:18:56.567044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.567025 2569 fs.go:135] Filesystem UUIDs: map[55c2b9c1-bd82-474c-9d7d-8612c3990f39:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9a6c507f-68b9-4d12-96d5-62f626b938e1:/dev/nvme0n1p4] Jul 21 08:18:56.567121 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.567045 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 08:18:56.572098 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.572069 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 08:18:56.573418 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.573299 2569 manager.go:216] Machine: {Timestamp:2026-07-21 08:18:56.571463431 +0000 UTC m=+0.333144615 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096873 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c6865a329dd19801e2df06f567f7a SystemUUID:ec2c6865-a329-dd19-801e-2df06f567f7a BootID:97c380d1-e9d2-4279-88f1-71eb9f30410f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:a7:53:78:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:a7:53:78:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:bc:6a:80:3f:bf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 08:18:56.573418 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.573413 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 08:18:56.573568 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.573556 2569 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 08:18:56.576573 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.576535 2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 08:18:56.576716 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.576575 2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-97.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 08:18:56.577251 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.577241 2569 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 08:18:56.577303 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.577254 2569 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 08:18:56.577303 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.577263 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 08:18:56.578649 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.578635 2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 08:18:56.579873 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.579860 2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 08:18:56.583218 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583201 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 21 08:18:56.583365 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583354 2569 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 08:18:56.583398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583387 2569 kubelet.go:486] "Attempting to sync node with API server" Jul 21 08:18:56.583433 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583398 2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 08:18:56.583433 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583412 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 08:18:56.583433 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583422 2569 kubelet.go:398] "Adding apiserver pod source" Jul 21 08:18:56.583433 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583432 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 08:18:56.583875 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583849 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 08:18:56.583875 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.583874 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 08:18:56.586093 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.586068 2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 08:18:56.588153 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.588138 2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 08:18:56.588228 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.588169 2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 08:18:56.589236 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589220 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 08:18:56.589236 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589237 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589244 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589250 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589258 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589263 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589287 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589292 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589300 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589305 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589314 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 08:18:56.589340 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589319 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 08:18:56.590004 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589981 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 08:18:56.590004 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.589994 2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 08:18:56.591887 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.591867 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8vlp" Jul 21 08:18:56.593749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.593731 2569 server.go:1268] "Started kubelet" Jul 21 08:18:56.594016 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.593995 2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-97.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 08:18:56.594104 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.594052 2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 08:18:56.594104 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.594098 2569 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 08:18:56.594285 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.594240 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 08:18:56.594425 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.594386 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 08:18:56.594485 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.594436 2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-97.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 08:18:56.594853 ip-10-0-142-97 systemd[1]: Started Kubernetes Kubelet. Jul 21 08:18:56.595979 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.595883 2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 08:18:56.596402 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.596386 2569 server.go:310] "Adding debug handlers to kubelet server" Jul 21 08:18:56.598076 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.598058 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b8vlp" Jul 21 08:18:56.598762 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.598730 2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 08:18:56.599889 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.598871 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-97.ec2.internal.18c440322d0988b6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-97.ec2.internal,UID:ip-10-0-142-97.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-97.ec2.internal,},FirstTimestamp:2026-07-21 08:18:56.593701046 +0000 UTC m=+0.355382224,LastTimestamp:2026-07-21 08:18:56.593701046 +0000 UTC m=+0.355382224,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-97.ec2.internal,}" Jul 21 08:18:56.602228 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.602210 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 08:18:56.602335 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.602231 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 08:18:56.602335 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.602295 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:56.602952 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.602932 2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 08:18:56.602952 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.602937 2569 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 08:18:56.603069 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.602971 2569 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 08:18:56.603392 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603286 2569 reconstruct.go:97] "Volume reconstruction finished" Jul 21 08:18:56.603485 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603416 2569 reconciler.go:29] "Reconciler: start to sync state" Jul 21 08:18:56.603485 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603419 2569 factory.go:55] Registering systemd factory Jul 21 08:18:56.603485 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603443 2569 factory.go:223] Registration of the systemd container factory successfully Jul 21 08:18:56.603704 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603693 2569 factory.go:153] Registering CRI-O factory Jul 21 08:18:56.603737 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603707 2569 factory.go:223] Registration of the crio container factory successfully Jul 21 08:18:56.603773 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603763 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 08:18:56.603807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603793 2569 factory.go:103] Registering Raw factory Jul 21 08:18:56.603843 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.603811 2569 manager.go:1196] Started watching for new ooms in manager Jul 21 08:18:56.604157 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.604142 2569 manager.go:349] Starting recovery of all containers Jul 21 08:18:56.611679 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.611657 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:18:56.615566 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.615542 2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.616722 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.616700 2569 manager.go:354] Recovery completed Jul 21 08:18:56.618079 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.618050 2569 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 21 08:18:56.621191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.621173 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.623335 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623317 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.623392 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623351 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.623392 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623361 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.623967 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623950 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 08:18:56.623967 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623965 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 08:18:56.624072 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623970 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.624072 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.623982 2569 state_mem.go:36] "Initialized new in-memory state store" Jul 21 08:18:56.624693 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.624679 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.624749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.624709 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.624749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.624723 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.625380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.625368 2569 policy_none.go:49] "None policy: Start" Jul 21 08:18:56.625418 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.625385 2569 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 08:18:56.625418 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.625396 2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 08:18:56.626422 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.626412 2569 policy_none.go:47] "Start" Jul 21 08:18:56.672051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672033 2569 manager.go:337] "Starting Device Plugin manager" Jul 21 08:18:56.672139 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.672078 2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 08:18:56.672139 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672091 2569 server.go:87] "Starting device plugin registration server" Jul 21 08:18:56.672419 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672406 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 08:18:56.672460 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672421 2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 08:18:56.672553 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672528 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 08:18:56.672636 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672618 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 08:18:56.672636 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.672634 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 08:18:56.673386 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.673208 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 08:18:56.673460 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.673415 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:56.727778 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.727739 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 08:18:56.729226 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.729203 2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 08:18:56.729323 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.729245 2569 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 08:18:56.729323 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.729289 2569 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 08:18:56.729409 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.729330 2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 08:18:56.731368 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.731349 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:18:56.773509 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.773425 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.774457 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.774438 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.774578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.774476 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.774578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.774490 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.774578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.774521 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.782837 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.782814 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.782926 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.782845 2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-97.ec2.internal\": node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:56.795489 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.795463 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:56.830465 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.830432 2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal"] Jul 21 08:18:56.830546 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.830517 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.832045 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.832030 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.832116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.832058 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.832116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.832067 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.833328 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.833316 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.833425 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.833411 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.833477 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.833441 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.834034 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834017 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.834102 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834042 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.834102 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834052 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.834199 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834149 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.834199 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834174 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.834199 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.834189 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.835147 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.835130 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.835234 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.835162 2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 08:18:56.836020 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.836004 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientMemory" Jul 21 08:18:56.836086 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.836034 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 08:18:56.836086 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.836048 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeHasSufficientPID" Jul 21 08:18:56.856199 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.856170 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.862664 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.862643 2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-97.ec2.internal\" not found" node="ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.896351 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.896321 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:56.905393 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.905367 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.905474 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.905406 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.905474 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:56.905424 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3259959531b2cae4a95970f39ca273a7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"3259959531b2cae4a95970f39ca273a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:56.996679 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:56.996641 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:57.006106 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006081 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.006157 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006109 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.006157 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006128 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3259959531b2cae4a95970f39ca273a7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"3259959531b2cae4a95970f39ca273a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.006221 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006167 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3259959531b2cae4a95970f39ca273a7-config\") pod \"kube-apiserver-proxy-ip-10-0-142-97.ec2.internal\" (UID: \"3259959531b2cae4a95970f39ca273a7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.006221 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006200 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.006301 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.006200 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3fe2c8d6a8514d87ee3bfa611989887-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal\" (UID: \"c3fe2c8d6a8514d87ee3bfa611989887\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.097599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:57.097537 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:57.161126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.161090 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.166144 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:57.166123 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:57.198116 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:57.198074 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:57.298626 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:57.298588 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:58.497637 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:57.399142 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:58.598215 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:58.598174 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:58.641734 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.641701 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 08:18:58.642283 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.641849 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 08:18:58.642283 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.641896 2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 08:18:58.698492 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:58.698463 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:58.701522 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.701464 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 08:13:57 +0000 UTC" deadline="2028-03-01 12:09:25.474433978 +0000 UTC" Jul 21 08:18:58.701658 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.701540 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14139h50m26.772899611s" Jul 21 08:18:58.793366 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:58.793328 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3fe2c8d6a8514d87ee3bfa611989887.slice/crio-c7c43e3a3367a548ec964272ef37044f6f336785ba750b02a2060709bf7d4b1a WatchSource:0}: Error finding container c7c43e3a3367a548ec964272ef37044f6f336785ba750b02a2060709bf7d4b1a: Status 404 returned error can't find the container with id c7c43e3a3367a548ec964272ef37044f6f336785ba750b02a2060709bf7d4b1a Jul 21 08:18:58.794664 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:18:58.794640 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3259959531b2cae4a95970f39ca273a7.slice/crio-d6fab58b45f796a471c50f4ad7debe229e0603d9204bcbe252b336d84227cbc6 WatchSource:0}: Error finding container d6fab58b45f796a471c50f4ad7debe229e0603d9204bcbe252b336d84227cbc6: Status 404 returned error can't find the container with id d6fab58b45f796a471c50f4ad7debe229e0603d9204bcbe252b336d84227cbc6 Jul 21 08:18:58.798533 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:58.798509 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:58.799443 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.799429 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:18:58.831250 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.831197 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" event={"ID":"3259959531b2cae4a95970f39ca273a7","Type":"ContainerStarted","Data":"d6fab58b45f796a471c50f4ad7debe229e0603d9204bcbe252b336d84227cbc6"} Jul 21 08:18:58.832235 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:58.832213 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"c3fe2c8d6a8514d87ee3bfa611989887","Type":"ContainerStarted","Data":"c7c43e3a3367a548ec964272ef37044f6f336785ba750b02a2060709bf7d4b1a"} Jul 21 08:18:58.899499 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:58.899459 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:59.000014 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:58.999981 2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-97.ec2.internal\" not found" Jul 21 08:18:59.077208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.077124 2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:18:59.100940 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.100907 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" Jul 21 08:18:59.110637 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.110608 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 08:18:59.112513 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.112496 2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" Jul 21 08:18:59.120732 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.120711 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 08:18:59.212642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.212607 2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:18:59.628614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.628579 2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:18:59.683526 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.683493 2569 apiserver.go:52] "Watching apiserver" Jul 21 08:18:59.689561 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.689529 2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 08:18:59.690031 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.690004 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-9b88v","openshift-multus/multus-pt8k8","openshift-multus/network-metrics-daemon-zfdwm","openshift-network-diagnostics/network-check-target-vrc2v","openshift-network-operator/iptables-alerter-wtbq6","openshift-ovn-kubernetes/ovnkube-node-kt5wl","kube-system/global-pull-secret-syncer-jkrrd","kube-system/konnectivity-agent-r7kh5","kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj","openshift-dns/node-resolver-hpg6q","openshift-image-registry/node-ca-tmmt9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal","openshift-multus/multus-additional-cni-plugins-mqzlz"] Jul 21 08:18:59.692649 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.692623 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.692792 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.692717 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:18:59.695618 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.695594 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.697660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.697635 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.697773 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.697739 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:18:59.697934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.697913 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.698055 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.697956 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x5ftn\"" Jul 21 08:18:59.698055 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.698041 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.700655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.700628 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:18:59.700779 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.700714 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:18:59.701904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.701885 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 08:18:59.702942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.702927 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.705955 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.705540 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.706042 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.705987 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.706088 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.706039 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vztbx\"" Jul 21 08:18:59.706370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.706338 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 08:18:59.706486 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.706447 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.709198 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709170 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.709316 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709203 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 08:18:59.709316 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709207 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 08:18:59.709428 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709392 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.709639 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709618 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 08:18:59.709752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709653 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 08:18:59.710098 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.709872 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f4fp4\"" Jul 21 08:18:59.711110 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.711094 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.712816 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.712797 2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 08:18:59.713242 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713222 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h9khx\"" Jul 21 08:18:59.713332 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713310 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.713404 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713350 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.713404 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713398 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 08:18:59.713600 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713568 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 08:18:59.713673 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713599 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 08:18:59.713765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713697 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-dbus\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.713765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713729 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-hosts-file\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.713765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713759 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:18:59.713929 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-iptables-alerter-script\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.713929 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713886 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-host-slash\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.714020 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.713970 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.714107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714042 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-tmp-dir\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.714107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714072 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.714208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714118 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2xq4\" (UniqueName: \"kubernetes.io/projected/d71ad86f-e6b6-45b0-87dc-63af669ddfff-kube-api-access-t2xq4\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.714208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714145 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzft7\" (UniqueName: \"kubernetes.io/projected/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-kube-api-access-vzft7\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.714208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714172 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-kubelet-config\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.714341 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp8rg\" (UniqueName: \"kubernetes.io/projected/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-kube-api-access-hp8rg\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.714400 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.714383 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.716752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.716626 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 08:18:59.716752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.716709 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-52k75\"" Jul 21 08:18:59.716935 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.716900 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 08:18:59.718772 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.718749 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.721663 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.721084 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.721663 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.721206 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b6gpp\"" Jul 21 08:18:59.721663 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.721507 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 08:18:59.723956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.723321 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 08:18:59.723956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.723364 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jb2fk\"" Jul 21 08:18:59.723956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.723537 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.723956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.723636 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.725615 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.725596 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.727594 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.727577 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.727673 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.727618 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pncr8\"" Jul 21 08:18:59.727773 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.727738 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.728029 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.728011 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.730483 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.730456 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhbf2\"" Jul 21 08:18:59.730788 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.730768 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 08:18:59.730788 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.730782 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 08:18:59.730916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.730770 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 08:18:59.734246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.734226 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2hgxn" Jul 21 08:18:59.741764 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.741740 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2hgxn" Jul 21 08:18:59.802271 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.802241 2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 08:18:59.815146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815113 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.815146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815149 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-sys-fs\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e35a47c-9f4f-4988-abe0-c7575f829bea-host\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815217 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-os-release\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815252 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-bin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815284 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-host-slash\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-tmp-dir\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815340 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-ovn\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815361 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c583c41-6a5e-478d-84b6-a247cb285603-ovn-node-metrics-cert\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.815375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815374 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-host-slash\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815389 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815503 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p8wv\" (UniqueName: \"kubernetes.io/projected/3e35a47c-9f4f-4988-abe0-c7575f829bea-kube-api-access-4p8wv\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815533 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-multus-daemon-config\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815555 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zdfw\" (UniqueName: \"kubernetes.io/projected/10084262-4d04-4cd6-884b-938faa34cefb-kube-api-access-4zdfw\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzft7\" (UniqueName: \"kubernetes.io/projected/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-kube-api-access-vzft7\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815630 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-device-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815701 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-tmp-dir\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815724 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-systemd\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.815775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815766 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-host\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815825 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-dbus\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-systemd-units\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815896 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-system-cni-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-modprobe-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815954 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-kubelet\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.815982 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68dvw\" (UniqueName: \"kubernetes.io/projected/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kube-api-access-68dvw\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816005 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-sys\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816029 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e35a47c-9f4f-4988-abe0-c7575f829bea-serviceca\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816052 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-os-release\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816063 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-dbus\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816075 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gxvv\" (UniqueName: \"kubernetes.io/projected/a8834bb4-ae86-4d20-b4d8-25f8cd803703-kube-api-access-2gxvv\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816109 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-k8s-cni-cncf-io\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816146 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-kubelet-config\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816168 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-systemd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816183 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-script-lib\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816197 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysconfig\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-kubelet-config\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816211 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-binary-copy\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816248 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5291dae6-6ea3-4477-92a1-25b390c568e3-konnectivity-ca\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816273 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-cni-binary-copy\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816290 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-netns\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816312 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-kubelet\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816335 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-etc-kubernetes\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816352 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-tmp\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816369 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816391 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-node-log\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816423 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-config\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-etc-selinux\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816463 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-lib-modules\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816487 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.816919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816512 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816528 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-log-socket\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816548 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfkb\" (UniqueName: \"kubernetes.io/projected/4c583c41-6a5e-478d-84b6-a247cb285603-kube-api-access-4vfkb\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816582 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-conf\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816650 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-hosts-file\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.816660 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-bin\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.816742 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:00.316724236 +0000 UTC m=+2.979993440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816772 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-hosts-file\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816793 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-env-overrides\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816819 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-run\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816838 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-system-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816888 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-conf-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816916 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-etc-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816939 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.817675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.816975 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5291dae6-6ea3-4477-92a1-25b390c568e3-agent-certs\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817002 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-multus-certs\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817036 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-iptables-alerter-script\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817056 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817073 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2xq4\" (UniqueName: \"kubernetes.io/projected/d71ad86f-e6b6-45b0-87dc-63af669ddfff-kube-api-access-t2xq4\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817090 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-netns\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817111 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-socket-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817134 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-kubernetes\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817157 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cnibin\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817178 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-cnibin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817217 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp8rg\" (UniqueName: \"kubernetes.io/projected/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-kube-api-access-hp8rg\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817238 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-netd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817257 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-tuned\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.817272 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817280 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpdqw\" (UniqueName: \"kubernetes.io/projected/5b57a875-5fcd-4463-9383-fa63a52ac191-kube-api-access-wpdqw\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817304 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-slash\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818243 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.817339 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:00.317321307 +0000 UTC m=+2.980590517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817395 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-registration-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817419 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-socket-dir-parent\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817453 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-multus\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817480 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-hostroot\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-var-lib-kubelet\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817539 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-var-lib-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817564 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.818771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.817667 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-iptables-alerter-script\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.826128 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.826086 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 08:18:59.828599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.828555 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:18:59.828599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.828580 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:18:59.828599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.828594 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:18:59.828871 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:18:59.828664 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:00.328644743 +0000 UTC m=+2.991913946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:18:59.830596 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.830570 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzft7\" (UniqueName: \"kubernetes.io/projected/4ec3bb79-30f8-483b-b9a6-03c7076e6a33-kube-api-access-vzft7\") pod \"iptables-alerter-wtbq6\" (UID: \"4ec3bb79-30f8-483b-b9a6-03c7076e6a33\") " pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:18:59.831444 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.831408 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp8rg\" (UniqueName: \"kubernetes.io/projected/4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4-kube-api-access-hp8rg\") pod \"node-resolver-hpg6q\" (UID: \"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4\") " pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:18:59.831544 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.831522 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2xq4\" (UniqueName: \"kubernetes.io/projected/d71ad86f-e6b6-45b0-87dc-63af669ddfff-kube-api-access-t2xq4\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:18:59.918823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918741 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-systemd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.918823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918773 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-script-lib\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.918823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysconfig\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918870 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-systemd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918934 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysconfig\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.918967 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-binary-copy\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919008 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5291dae6-6ea3-4477-92a1-25b390c568e3-konnectivity-ca\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919033 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-cni-binary-copy\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.919089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919062 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-netns\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919094 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-kubelet\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919119 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-etc-kubernetes\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919141 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-tmp\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919180 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-node-log\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919202 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-config\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919244 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-etc-selinux\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919269 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-lib-modules\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-log-socket\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919366 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfkb\" (UniqueName: \"kubernetes.io/projected/4c583c41-6a5e-478d-84b6-a247cb285603-kube-api-access-4vfkb\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919390 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-conf\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919414 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-bin\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919436 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-env-overrides\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-run\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919490 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-script-lib\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919501 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-system-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919563 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-conf-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919586 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-etc-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919594 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-binary-copy\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919636 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.919655 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919641 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5291dae6-6ea3-4477-92a1-25b390c568e3-agent-certs\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919671 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-multus-certs\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919745 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-etc-selinux\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919773 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-run\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919786 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-netns\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919793 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-multus-certs\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919807 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-log-socket\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919913 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-socket-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919922 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-conf\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919968 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-system-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920010 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-conf-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920040 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-bin\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920071 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-etc-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920245 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920329 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-cni-binary-copy\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.920364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920363 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-netns\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920388 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-env-overrides\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920394 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-kubelet\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920429 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-etc-kubernetes\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920510 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-lib-modules\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.919810 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-socket-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920738 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-kubernetes\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920769 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cnibin\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920794 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-cnibin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920826 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-node-log\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920872 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-netns\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920889 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-cnibin\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.920959 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-kubernetes\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921001 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-cnibin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921031 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-netd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-tuned\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpdqw\" (UniqueName: \"kubernetes.io/projected/5b57a875-5fcd-4463-9383-fa63a52ac191-kube-api-access-wpdqw\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921109 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5291dae6-6ea3-4477-92a1-25b390c568e3-konnectivity-ca\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.921175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-slash\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921165 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-cni-netd\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921183 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921264 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c583c41-6a5e-478d-84b6-a247cb285603-ovnkube-config\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921281 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8834bb4-ae86-4d20-b4d8-25f8cd803703-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-registration-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921326 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-slash\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921354 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-run-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921398 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-socket-dir-parent\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921424 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-multus\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921451 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-hostroot\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921461 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-registration-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921493 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-var-lib-kubelet\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921503 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-multus\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921527 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-socket-dir-parent\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921565 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-hostroot\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921572 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-var-lib-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.921601 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.922883 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.922283 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-var-lib-kubelet\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.923192 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923170 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-tmp\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.923301 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923283 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.923363 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923296 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-var-lib-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.923415 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923371 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-sys-fs\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.923468 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923413 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-openvswitch\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.923468 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923431 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e35a47c-9f4f-4988-abe0-c7575f829bea-host\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.923468 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923465 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-sys-fs\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.923593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923499 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-os-release\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.923593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923520 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-bin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.923593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923551 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e35a47c-9f4f-4988-abe0-c7575f829bea-host\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.923593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923580 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923590 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-tuned\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923622 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-os-release\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923672 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-cni-bin\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923698 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-ovn\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923735 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c583c41-6a5e-478d-84b6-a247cb285603-ovn-node-metrics-cert\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923775 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-run-ovn\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.923828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923813 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923862 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p8wv\" (UniqueName: \"kubernetes.io/projected/3e35a47c-9f4f-4988-abe0-c7575f829bea-kube-api-access-4p8wv\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923893 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-multus-daemon-config\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923900 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5291dae6-6ea3-4477-92a1-25b390c568e3-agent-certs\") pod \"konnectivity-agent-r7kh5\" (UID: \"5291dae6-6ea3-4477-92a1-25b390c568e3\") " pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdfw\" (UniqueName: \"kubernetes.io/projected/10084262-4d04-4cd6-884b-938faa34cefb-kube-api-access-4zdfw\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.923989 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924043 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-device-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924066 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-systemd\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924093 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-host\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924126 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924123 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924064 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-sysctl-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924203 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-systemd-units\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924228 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-system-cni-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924257 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-modprobe-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924287 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-kubelet\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68dvw\" (UniqueName: \"kubernetes.io/projected/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kube-api-access-68dvw\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924344 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-sys\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e35a47c-9f4f-4988-abe0-c7575f829bea-serviceca\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924463 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-os-release\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924488 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gxvv\" (UniqueName: \"kubernetes.io/projected/a8834bb4-ae86-4d20-b4d8-25f8cd803703-kube-api-access-2gxvv\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924517 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-k8s-cni-cncf-io\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924566 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-systemd-units\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924516 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10084262-4d04-4cd6-884b-938faa34cefb-multus-daemon-config\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.924606 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924608 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-system-cni-dir\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.925326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924630 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-sys\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.925326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924710 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-multus-cni-dir\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.925326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924723 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-modprobe-d\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.925326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.924767 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-var-lib-kubelet\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.925326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925220 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c583c41-6a5e-478d-84b6-a247cb285603-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.925762 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925737 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c583c41-6a5e-478d-84b6-a247cb285603-ovn-node-metrics-cert\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.925899 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925762 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-host\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.925899 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925819 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10084262-4d04-4cd6-884b-938faa34cefb-host-run-k8s-cni-cncf-io\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.925899 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925820 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5b57a875-5fcd-4463-9383-fa63a52ac191-etc-systemd\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.925899 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925862 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-device-dir\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.926088 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.925923 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8834bb4-ae86-4d20-b4d8-25f8cd803703-os-release\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.926088 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.926006 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3e35a47c-9f4f-4988-abe0-c7575f829bea-serviceca\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:18:59.931191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.931169 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpdqw\" (UniqueName: \"kubernetes.io/projected/5b57a875-5fcd-4463-9383-fa63a52ac191-kube-api-access-wpdqw\") pod \"tuned-9b88v\" (UID: \"5b57a875-5fcd-4463-9383-fa63a52ac191\") " pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:18:59.931634 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.931603 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfkb\" (UniqueName: \"kubernetes.io/projected/4c583c41-6a5e-478d-84b6-a247cb285603-kube-api-access-4vfkb\") pod \"ovnkube-node-kt5wl\" (UID: \"4c583c41-6a5e-478d-84b6-a247cb285603\") " pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:18:59.933394 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.933373 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gxvv\" (UniqueName: \"kubernetes.io/projected/a8834bb4-ae86-4d20-b4d8-25f8cd803703-kube-api-access-2gxvv\") pod \"multus-additional-cni-plugins-mqzlz\" (UID: \"a8834bb4-ae86-4d20-b4d8-25f8cd803703\") " pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:18:59.933481 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.933387 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68dvw\" (UniqueName: \"kubernetes.io/projected/0f1b11f5-1cc6-450e-8b4f-ca84bf636da3-kube-api-access-68dvw\") pod \"aws-ebs-csi-driver-node-dl5fj\" (UID: \"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:18:59.933827 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.933805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zdfw\" (UniqueName: \"kubernetes.io/projected/10084262-4d04-4cd6-884b-938faa34cefb-kube-api-access-4zdfw\") pod \"multus-pt8k8\" (UID: \"10084262-4d04-4cd6-884b-938faa34cefb\") " pod="openshift-multus/multus-pt8k8" Jul 21 08:18:59.933916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:18:59.933894 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p8wv\" (UniqueName: \"kubernetes.io/projected/3e35a47c-9f4f-4988-abe0-c7575f829bea-kube-api-access-4p8wv\") pod \"node-ca-tmmt9\" (UID: \"3e35a47c-9f4f-4988-abe0-c7575f829bea\") " pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:19:00.008012 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.007965 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpg6q" Jul 21 08:19:00.021041 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.021012 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wtbq6" Jul 21 08:19:00.030118 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.028884 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:00.030334 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.030318 2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 08:19:00.033212 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.033189 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" Jul 21 08:19:00.039872 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.039832 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:19:00.045715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.045673 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pt8k8" Jul 21 08:19:00.053549 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.053520 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" Jul 21 08:19:00.060375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.060349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9b88v" Jul 21 08:19:00.068179 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.068154 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tmmt9" Jul 21 08:19:00.327048 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.326960 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:00.327048 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.327008 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:00.327271 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.327131 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:00.327271 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.327131 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:00.327271 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.327199 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:01.327179788 +0000 UTC m=+3.990449004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:00.327271 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.327219 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:01.327208414 +0000 UTC m=+3.990477624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:00.428382 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.428342 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:00.428574 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.428549 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:00.428574 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.428569 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:00.428695 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.428584 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:00.428695 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.428644 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:01.428626455 +0000 UTC m=+4.091895658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:00.745130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.745045 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 08:13:59 +0000 UTC" deadline="2028-02-16 18:34:02.44383967 +0000 UTC" Jul 21 08:19:00.745130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.745119 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13810h15m1.69872324s" Jul 21 08:19:00.779194 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:00.779153 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c583c41_6a5e_478d_84b6_a247cb285603.slice/crio-862ee3b6c43a53cfab493924bfd2fd1ff601d8a21a566385f5ad2e2b0c4443b2 WatchSource:0}: Error finding container 862ee3b6c43a53cfab493924bfd2fd1ff601d8a21a566385f5ad2e2b0c4443b2: Status 404 returned error can't find the container with id 862ee3b6c43a53cfab493924bfd2fd1ff601d8a21a566385f5ad2e2b0c4443b2 Jul 21 08:19:00.783282 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:00.783168 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b57a875_5fcd_4463_9383_fa63a52ac191.slice/crio-e987a472c69219ecaf257770cc27481778ae0d43e3c1da7942a0b5a8eda5db17 WatchSource:0}: Error finding container e987a472c69219ecaf257770cc27481778ae0d43e3c1da7942a0b5a8eda5db17: Status 404 returned error can't find the container with id e987a472c69219ecaf257770cc27481778ae0d43e3c1da7942a0b5a8eda5db17 Jul 21 08:19:00.786318 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:00.786288 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec3bb79_30f8_483b_b9a6_03c7076e6a33.slice/crio-5ba8483e62bd5359774d71ff85c32a994f5a5a66c4c2e14a1c71016dfd8768d1 WatchSource:0}: Error finding container 5ba8483e62bd5359774d71ff85c32a994f5a5a66c4c2e14a1c71016dfd8768d1: Status 404 returned error can't find the container with id 5ba8483e62bd5359774d71ff85c32a994f5a5a66c4c2e14a1c71016dfd8768d1 Jul 21 08:19:00.790277 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:00.789954 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1b11f5_1cc6_450e_8b4f_ca84bf636da3.slice/crio-31dc4220ff754972c7df802ae3b8af6d4f55adc381e521f65b1d8420b0785b44 WatchSource:0}: Error finding container 31dc4220ff754972c7df802ae3b8af6d4f55adc381e521f65b1d8420b0785b44: Status 404 returned error can't find the container with id 31dc4220ff754972c7df802ae3b8af6d4f55adc381e521f65b1d8420b0785b44 Jul 21 08:19:00.828329 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.828298 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:00.828465 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:00.828438 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:00.838916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.838875 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerStarted","Data":"1a4265e5cf96fa92b9daebb590344048c88e6ad3de4cfe9cd11585e5284c4d66"} Jul 21 08:19:00.840573 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.840542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wtbq6" event={"ID":"4ec3bb79-30f8-483b-b9a6-03c7076e6a33","Type":"ContainerStarted","Data":"5ba8483e62bd5359774d71ff85c32a994f5a5a66c4c2e14a1c71016dfd8768d1"} Jul 21 08:19:00.841752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.841724 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9b88v" event={"ID":"5b57a875-5fcd-4463-9383-fa63a52ac191","Type":"ContainerStarted","Data":"e987a472c69219ecaf257770cc27481778ae0d43e3c1da7942a0b5a8eda5db17"} Jul 21 08:19:00.842979 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.842948 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"862ee3b6c43a53cfab493924bfd2fd1ff601d8a21a566385f5ad2e2b0c4443b2"} Jul 21 08:19:00.844057 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.844036 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r7kh5" event={"ID":"5291dae6-6ea3-4477-92a1-25b390c568e3","Type":"ContainerStarted","Data":"c0697d9152d66ade92807de0f928f28fadd49b26f2835f4e826fae1a866fa20f"} Jul 21 08:19:00.845384 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.845356 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpg6q" event={"ID":"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4","Type":"ContainerStarted","Data":"79c90459954a4aff8ca61bbc78f41f8a49b06fd8fb758812893994985f9dd6d9"} Jul 21 08:19:00.846531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.846511 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" event={"ID":"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3","Type":"ContainerStarted","Data":"31dc4220ff754972c7df802ae3b8af6d4f55adc381e521f65b1d8420b0785b44"} Jul 21 08:19:00.847928 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.847879 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pt8k8" event={"ID":"10084262-4d04-4cd6-884b-938faa34cefb","Type":"ContainerStarted","Data":"89523c4f385e26308ce92d9873f27e3663a3622a07c79ec60bc89ccad5f7f53e"} Jul 21 08:19:00.849032 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:00.849012 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tmmt9" event={"ID":"3e35a47c-9f4f-4988-abe0-c7575f829bea","Type":"ContainerStarted","Data":"91f7d091d21dc5c9c9f32d45aa2e33f7c0ec496f127744b31770d2c1e35e21ae"} Jul 21 08:19:01.335162 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.334835 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:01.335162 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.335145 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:01.335416 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.335159 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:01.335416 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.335230 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:03.335207089 +0000 UTC m=+5.998476295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:01.335416 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.335281 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:01.335416 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.335331 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:03.335315661 +0000 UTC m=+5.998584869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:01.436364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.436318 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:01.436557 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.436510 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:01.436557 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.436533 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:01.436557 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.436547 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:01.436766 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.436620 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:03.436591779 +0000 UTC m=+6.099860985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:01.746089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.745961 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 08:13:59 +0000 UTC" deadline="2028-03-04 09:25:54.851713352 +0000 UTC" Jul 21 08:19:01.746089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.746033 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14209h6m53.105684592s" Jul 21 08:19:01.832071 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.831307 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:01.832071 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.831448 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:01.832071 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.831899 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:01.832071 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:01.832014 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:01.877288 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.877248 2569 generic.go:356] "Generic (PLEG): container finished" podID="c3fe2c8d6a8514d87ee3bfa611989887" containerID="4e1d7b66ae2750d1994644f8feb58d59bfdf1a85a39d61a6c87207a2e43c518d" exitCode=0 Jul 21 08:19:01.878226 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.878157 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"c3fe2c8d6a8514d87ee3bfa611989887","Type":"ContainerDied","Data":"4e1d7b66ae2750d1994644f8feb58d59bfdf1a85a39d61a6c87207a2e43c518d"} Jul 21 08:19:01.896564 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:01.896525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" event={"ID":"3259959531b2cae4a95970f39ca273a7","Type":"ContainerStarted","Data":"1e8e4c698a736e74a09ed67af3d35a39b9021e886cddc10e52d8a349d2d5be6b"} Jul 21 08:19:02.828088 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:02.828037 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:02.828589 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:02.828164 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:02.916402 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:02.916353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" event={"ID":"c3fe2c8d6a8514d87ee3bfa611989887","Type":"ContainerStarted","Data":"a0e50dba245fc6b40184109f2ff89a53fff0abbfb2d361fa2501a33a6161d12c"} Jul 21 08:19:02.929603 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:02.929541 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-97.ec2.internal" podStartSLOduration=3.929517547 podStartE2EDuration="3.929517547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:01.910421161 +0000 UTC m=+4.573690388" watchObservedRunningTime="2026-07-21 08:19:02.929517547 +0000 UTC m=+5.592786755" Jul 21 08:19:03.352766 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:03.352725 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:03.352964 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:03.352855 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:03.353080 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.353003 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:03.353080 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.353063 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:07.353044934 +0000 UTC m=+10.016314140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:03.353561 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.353537 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:03.353662 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.353597 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:07.353580119 +0000 UTC m=+10.016849327 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:03.453627 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:03.453587 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:03.453877 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.453817 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:03.453877 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.453841 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:03.453877 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.453855 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:03.454151 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.453920 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:07.453900178 +0000 UTC m=+10.117169403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:03.830079 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:03.829869 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:03.830079 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.830019 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:03.830647 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:03.830544 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:03.840527 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:03.840479 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:04.828669 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:04.828632 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:04.828864 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:04.828772 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:05.830835 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:05.828291 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:05.830835 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:05.828428 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:05.830835 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:05.828850 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:05.830835 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:05.828952 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:06.828372 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:06.828336 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:06.828576 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:06.828476 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:07.388145 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:07.388098 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:07.388145 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:07.388147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:07.388634 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.388261 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:07.388634 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.388277 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:07.388634 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.388322 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:15.388308109 +0000 UTC m=+18.051577315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:07.388634 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.388346 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:15.388328264 +0000 UTC m=+18.051597472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:07.488751 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:07.488709 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:07.488940 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.488862 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:07.488940 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.488886 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:07.488940 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.488900 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:07.489107 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.488961 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:15.488944929 +0000 UTC m=+18.152214133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:07.829799 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:07.829718 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:07.830090 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.829901 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:07.830090 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:07.829999 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:07.830256 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:07.830136 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:08.828925 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:08.828744 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:08.828925 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:08.828885 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:09.829456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:09.828837 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:09.829456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:09.828885 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:09.829456 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:09.828988 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:09.829456 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:09.829399 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:10.828039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:10.828004 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:10.828215 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:10.828135 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:11.827997 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:11.827957 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:11.828485 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:11.828107 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:11.828485 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:11.828160 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:11.828485 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:11.828276 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:12.828992 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:12.828951 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:12.829403 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:12.829069 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:13.828816 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:13.828770 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:13.829007 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:13.828909 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:13.829268 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:13.828770 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:13.829560 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:13.829328 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:14.828699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:14.828645 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:14.828905 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:14.828805 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:15.452408 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:15.452365 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:15.452898 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:15.452459 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:15.452898 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.452530 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:15.452898 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.452534 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:15.452898 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.452602 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:19:31.452584587 +0000 UTC m=+34.115853792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:15.452898 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.452623 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:19:31.452613148 +0000 UTC m=+34.115882351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:15.553900 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:15.553810 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:15.554073 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.554001 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:15.554073 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.554028 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:15.554073 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.554039 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:15.554217 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.554097 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:19:31.554080628 +0000 UTC m=+34.217349854 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:15.828360 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:15.828269 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:15.828519 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.828418 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:15.828519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:15.828450 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:15.828595 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:15.828535 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:16.828902 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:16.828859 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:16.829379 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:16.828992 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:17.829603 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:17.829565 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:17.830100 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:17.829699 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:17.830100 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:17.829723 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:17.830100 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:17.829867 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:18.828396 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:18.828359 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:18.828604 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:18.828487 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:19.831767 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.831742 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:19.832200 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.832068 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:19.832200 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:19.832179 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:19.832659 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:19.832633 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:19.955439 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.955105 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tmmt9" event={"ID":"3e35a47c-9f4f-4988-abe0-c7575f829bea","Type":"ContainerStarted","Data":"89c693682fd84cf17b12dba45cd1a36278046fb14ff015b192e966e7482c554f"} Jul 21 08:19:19.956982 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.956944 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerStarted","Data":"190cb489baadd1271760d21eef629bcc3d99ef7cd7e325cb83b2ba0eb8f5cfbb"} Jul 21 08:19:19.958410 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.958383 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9b88v" event={"ID":"5b57a875-5fcd-4463-9383-fa63a52ac191","Type":"ContainerStarted","Data":"b9e0b9e4c6815b5a4987f1d082abcce0b11ce4bf23ac5ce27b4b9e94769d4881"} Jul 21 08:19:19.960395 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.960362 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r7kh5" event={"ID":"5291dae6-6ea3-4477-92a1-25b390c568e3","Type":"ContainerStarted","Data":"c64054504af167c5608d32dfc7baf63c0966cfd1d0a861a789f04627be6b144c"} Jul 21 08:19:19.962156 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.962122 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpg6q" event={"ID":"4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4","Type":"ContainerStarted","Data":"07ced6cce9605e253693e8dcef804cf63fd446be7b484aa7f27022b97013f51e"} Jul 21 08:19:19.964602 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.964570 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" event={"ID":"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3","Type":"ContainerStarted","Data":"06527a055fbdf7d299e34153373b105c61416ec77e603713ceca4a9e33409247"} Jul 21 08:19:19.969212 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.968824 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pt8k8" event={"ID":"10084262-4d04-4cd6-884b-938faa34cefb","Type":"ContainerStarted","Data":"26890878519ee1c07312349e4cf4cf87fd92bb580cf03856238c164a6ca23a21"} Jul 21 08:19:19.969347 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.969182 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tmmt9" podStartSLOduration=4.161917382 podStartE2EDuration="22.969162749s" totalImagesPullingTime="18.807245367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:19.968702268 +0000 UTC m=+22.631971486" watchObservedRunningTime="2026-07-21 08:19:19.969162749 +0000 UTC m=+22.632431975" Jul 21 08:19:19.969347 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:19.969287 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-97.ec2.internal" podStartSLOduration=20.96927671 podStartE2EDuration="20.96927671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:02.93066377 +0000 UTC m=+5.593932996" watchObservedRunningTime="2026-07-21 08:19:19.96927671 +0000 UTC m=+22.632545937" Jul 21 08:19:20.001842 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.001778 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r7kh5" podStartSLOduration=8.731708526 podStartE2EDuration="23.001754732s" totalImagesPullingTime="14.270046206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:20.000779526 +0000 UTC m=+22.664048773" watchObservedRunningTime="2026-07-21 08:19:20.001754732 +0000 UTC m=+22.665023971" Jul 21 08:19:20.014139 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.014083 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hpg6q" podStartSLOduration=4.221492643 podStartE2EDuration="23.014065349s" totalImagesPullingTime="18.792572706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:20.01355504 +0000 UTC m=+22.676824265" watchObservedRunningTime="2026-07-21 08:19:20.014065349 +0000 UTC m=+22.677334574" Jul 21 08:19:20.030264 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.030199 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9b88v" podStartSLOduration=4.208531465 podStartE2EDuration="23.030174645s" totalImagesPullingTime="18.82164318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:20.029291271 +0000 UTC m=+22.692560497" watchObservedRunningTime="2026-07-21 08:19:20.030174645 +0000 UTC m=+22.693443969" Jul 21 08:19:20.047217 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.047167 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pt8k8" podStartSLOduration=4.183989381 podStartE2EDuration="23.047151757s" totalImagesPullingTime="18.863162376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:20.047127495 +0000 UTC m=+22.710396721" watchObservedRunningTime="2026-07-21 08:19:20.047151757 +0000 UTC m=+22.710420981" Jul 21 08:19:20.828456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.828281 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:20.828583 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:20.828556 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:20.972844 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.972809 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="190cb489baadd1271760d21eef629bcc3d99ef7cd7e325cb83b2ba0eb8f5cfbb" exitCode=0 Jul 21 08:19:20.973586 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.972887 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"190cb489baadd1271760d21eef629bcc3d99ef7cd7e325cb83b2ba0eb8f5cfbb"} Jul 21 08:19:20.975905 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.975859 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:19:20.976232 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976203 2569 generic.go:356] "Generic (PLEG): container finished" podID="4c583c41-6a5e-478d-84b6-a247cb285603" containerID="162aa0f618d105c984ae2e79efdb8a582ac7e025199307905ee94cbff772df52" exitCode=1 Jul 21 08:19:20.976344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976291 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"c23be34a62b216ab97fa5dfe3320650fbd630b262b88deffaf66e8d948c02838"} Jul 21 08:19:20.976395 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976376 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"b3af85af09d0580f62b4d7d7d54372e3bbc3a63530509e70bafa74ee49d364e3"} Jul 21 08:19:20.976430 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976417 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"f7ce3397217d3bb2458b4b8c9738b2e5e5d0146955a611922ad5748738803571"} Jul 21 08:19:20.976474 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976456 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"c298d3ceb18ee7d74a9d267ce6bf996ec02d06588accb5d31b3daa081a18a4e1"} Jul 21 08:19:20.976516 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976505 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerDied","Data":"162aa0f618d105c984ae2e79efdb8a582ac7e025199307905ee94cbff772df52"} Jul 21 08:19:20.976573 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:20.976559 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"485b5f1ab46c64b40784acaa1a1ab025d4e6e82e1cd7ce71aa523a775678d902"} Jul 21 08:19:21.014519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.014487 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 08:19:21.784518 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.784400 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T08:19:21.014510171Z","UUID":"bf1cf430-56af-4ef9-a86a-96e8a3ee4dca","Handler":null,"Name":"","Endpoint":""} Jul 21 08:19:21.786190 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.786169 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 08:19:21.786317 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.786206 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 08:19:21.828539 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.828504 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:21.828539 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.828522 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:21.828842 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:21.828633 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:21.828842 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:21.828790 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:21.980819 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.980778 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wtbq6" event={"ID":"4ec3bb79-30f8-483b-b9a6-03c7076e6a33","Type":"ContainerStarted","Data":"bc4a5312bcc143f13547980ca8d1c161fcbf881ea45fc734f10eb662376a4227"} Jul 21 08:19:21.982640 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.982614 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" event={"ID":"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3","Type":"ContainerStarted","Data":"d38e6f155c3efc5e84057437fd810cd0ff22d93548852233e21445a72ef414a3"} Jul 21 08:19:21.994192 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:21.994138 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wtbq6" podStartSLOduration=6.178035236 podStartE2EDuration="24.994122476s" totalImagesPullingTime="18.81608724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:21.993717676 +0000 UTC m=+24.656986899" watchObservedRunningTime="2026-07-21 08:19:21.994122476 +0000 UTC m=+24.657391700" Jul 21 08:19:22.828473 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:22.828435 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:22.828667 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:22.828541 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:22.988993 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:22.988964 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:19:22.989453 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:22.989425 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"69fece7b3d2d93806d77dfdcabf910dafc7775d389cf986c2c51289fbf74a9d0"} Jul 21 08:19:22.991769 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:22.991722 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" event={"ID":"0f1b11f5-1cc6-450e-8b4f-ca84bf636da3","Type":"ContainerStarted","Data":"7a501885acacf5bcadf61c3efc2a18fc6729aff92fbff1b99af0e7615daaa373"} Jul 21 08:19:23.008928 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.008866 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dl5fj" podStartSLOduration=4.888652665 podStartE2EDuration="26.008851188s" totalImagesPullingTime="21.120198523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:23.007724442 +0000 UTC m=+25.670993667" watchObservedRunningTime="2026-07-21 08:19:23.008851188 +0000 UTC m=+25.672120412" Jul 21 08:19:23.463144 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.463109 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:19:23.464052 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.464031 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:19:23.828403 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.828195 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:23.828580 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.828231 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:23.828580 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:23.828504 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:23.828580 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:23.828534 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:23.994613 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.994582 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:19:23.995049 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:23.994897 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r7kh5" Jul 21 08:19:24.828587 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:24.828548 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:24.828832 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:24.828800 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:25.000909 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:25.000881 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:19:25.001836 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:25.001371 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"4e9affa6fe19ee3e9dcc1ca6c90b1b4e0bccb20c416883fc941d09f85ae5a879"} Jul 21 08:19:25.002027 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:25.002005 2569 scope.go:117] "RemoveContainer" containerID="162aa0f618d105c984ae2e79efdb8a582ac7e025199307905ee94cbff772df52" Jul 21 08:19:25.829205 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:25.828977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:25.829384 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:25.828977 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:25.829384 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:25.829335 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:25.829521 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:25.829480 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:26.649151 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.649120 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkrrd"] Jul 21 08:19:26.649542 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.649230 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:26.649542 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:26.649319 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:26.652915 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.652885 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zfdwm"] Jul 21 08:19:26.653053 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.653011 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:26.653184 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:26.653161 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:26.653661 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.653626 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrc2v"] Jul 21 08:19:26.653774 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:26.653759 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:26.653875 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:26.653854 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:27.010331 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.010306 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:19:27.010741 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.010716 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" event={"ID":"4c583c41-6a5e-478d-84b6-a247cb285603","Type":"ContainerStarted","Data":"41b4a5f5f695f1933bf9988e01a74e8d736ca1f9efce436c017bddd6efaccef2"} Jul 21 08:19:27.010876 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.010814 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 21 08:19:27.010946 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.010928 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:27.011019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.011006 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:27.013480 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.013440 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerStarted","Data":"e0e7f37c16cdf1e33a80664dca4700aa85ea5c8725b030f0c0fca22a63e4da45"} Jul 21 08:19:27.032203 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.032177 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:27.032333 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.032297 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:27.035808 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:27.035765 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" podStartSLOduration=10.918589858 podStartE2EDuration="30.035746627s" totalImagesPullingTime="19.117156769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:27.034924716 +0000 UTC m=+29.698193978" watchObservedRunningTime="2026-07-21 08:19:27.035746627 +0000 UTC m=+29.699015844" Jul 21 08:19:28.017477 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.017438 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="e0e7f37c16cdf1e33a80664dca4700aa85ea5c8725b030f0c0fca22a63e4da45" exitCode=0 Jul 21 08:19:28.017857 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.017525 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"e0e7f37c16cdf1e33a80664dca4700aa85ea5c8725b030f0c0fca22a63e4da45"} Jul 21 08:19:28.017857 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.017619 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 21 08:19:28.828851 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.828634 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:28.828998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.828673 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:28.828998 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:28.828951 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:28.828998 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:28.828985 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:28.828998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:28.828672 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:28.829140 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:28.829061 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:29.022268 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.022237 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="e0b4c2087d476134cb9eb6fe13e5884bf1515557d7340fd49eba36112ead6e0e" exitCode=0 Jul 21 08:19:29.022630 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.022319 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"e0b4c2087d476134cb9eb6fe13e5884bf1515557d7340fd49eba36112ead6e0e"} Jul 21 08:19:29.022630 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.022448 2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 21 08:19:29.280525 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.280439 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:19:29.291665 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.291623 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" podUID="4c583c41-6a5e-478d-84b6-a247cb285603" containerName="ovnkube-controller" probeResult="failure" output="" Jul 21 08:19:29.301483 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:29.301449 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" podUID="4c583c41-6a5e-478d-84b6-a247cb285603" containerName="ovnkube-controller" probeResult="failure" output="" Jul 21 08:19:30.027470 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.027436 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="150b98de61fd82803bb5d38a8148f21a2f2cf45635cbf3217a184f07167e0e19" exitCode=0 Jul 21 08:19:30.027850 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.027519 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"150b98de61fd82803bb5d38a8148f21a2f2cf45635cbf3217a184f07167e0e19"} Jul 21 08:19:30.037951 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.037922 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" podUID="4c583c41-6a5e-478d-84b6-a247cb285603" containerName="ovnkube-controller" probeResult="failure" output="" Jul 21 08:19:30.828104 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.828072 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:30.828348 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.828074 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:30.828348 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:30.828179 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jkrrd" podUID="de0d6800-e8cd-4757-b2b7-9ee26630ec2a" Jul 21 08:19:30.828348 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:30.828074 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:30.828348 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:30.828262 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vrc2v" podUID="92a57fce-2394-4a70-9686-a0f6ed680a28" Jul 21 08:19:30.828480 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:30.828350 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:19:31.477176 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:31.477139 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:31.477802 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:31.477194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:31.477802 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.477316 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:31.477802 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.477390 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:20:03.477375583 +0000 UTC m=+66.140644786 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 08:19:31.477802 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.477320 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:31.477802 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.477472 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret podName:de0d6800-e8cd-4757-b2b7-9ee26630ec2a nodeName:}" failed. No retries permitted until 2026-07-21 08:20:03.47745392 +0000 UTC m=+66.140723125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret") pod "global-pull-secret-syncer-jkrrd" (UID: "de0d6800-e8cd-4757-b2b7-9ee26630ec2a") : object "kube-system"/"original-pull-secret" not registered Jul 21 08:19:31.577802 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:31.577765 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:31.578046 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.578019 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 08:19:31.578046 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.578050 2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 08:19:31.578219 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.578063 2569 projected.go:196] Error preparing data for projected volume kube-api-access-h7n5j for pod openshift-network-diagnostics/network-check-target-vrc2v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:31.578219 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:31.578161 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j podName:92a57fce-2394-4a70-9686-a0f6ed680a28 nodeName:}" failed. No retries permitted until 2026-07-21 08:20:03.578110535 +0000 UTC m=+66.241379738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h7n5j" (UniqueName: "kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j") pod "network-check-target-vrc2v" (UID: "92a57fce-2394-4a70-9686-a0f6ed680a28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 08:19:32.689725 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.689672 2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-97.ec2.internal" event="NodeReady" Jul 21 08:19:32.690324 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.689853 2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 08:19:32.729791 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.729760 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l7qwx"] Jul 21 08:19:32.734543 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.734512 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-htv2f"] Jul 21 08:19:32.734721 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.734639 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.736765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.736738 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 08:19:32.737217 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.737200 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jw7kl\"" Jul 21 08:19:32.737328 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.737204 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 08:19:32.738489 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.738467 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.740946 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.740574 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 08:19:32.740946 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.740724 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 08:19:32.740946 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.740775 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 08:19:32.740946 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.740724 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xmzts\"" Jul 21 08:19:32.744591 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.744570 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7qwx"] Jul 21 08:19:32.744712 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.744630 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htv2f"] Jul 21 08:19:32.828702 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.828643 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:19:32.828702 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.828666 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:19:32.828702 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.828694 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:19:32.831448 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831423 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 08:19:32.831594 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831450 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bh656\"" Jul 21 08:19:32.831594 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831504 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 08:19:32.831776 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831747 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 08:19:32.831942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831926 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4qzvw\"" Jul 21 08:19:32.831994 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.831967 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 08:19:32.888294 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888253 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19415ce2-6c93-40de-9e76-02f21f6dd80b-tmp-dir\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.888497 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888320 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rw9v\" (UniqueName: \"kubernetes.io/projected/b2f25635-fec1-46ac-9678-4ba245329c84-kube-api-access-6rw9v\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.888497 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888414 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.888497 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888470 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f25635-fec1-46ac-9678-4ba245329c84-cert\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.888635 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888507 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19415ce2-6c93-40de-9e76-02f21f6dd80b-config-volume\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.888635 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.888534 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwvs\" (UniqueName: \"kubernetes.io/projected/19415ce2-6c93-40de-9e76-02f21f6dd80b-kube-api-access-4qwvs\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.989518 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989427 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.989518 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989472 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f25635-fec1-46ac-9678-4ba245329c84-cert\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.989518 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989519 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19415ce2-6c93-40de-9e76-02f21f6dd80b-config-volume\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.989831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989551 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwvs\" (UniqueName: \"kubernetes.io/projected/19415ce2-6c93-40de-9e76-02f21f6dd80b-kube-api-access-4qwvs\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.989831 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:32.989594 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:32.989831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989612 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19415ce2-6c93-40de-9e76-02f21f6dd80b-tmp-dir\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.989831 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:32.989669 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:19:33.489647187 +0000 UTC m=+36.152916406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:32.989831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.989824 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rw9v\" (UniqueName: \"kubernetes.io/projected/b2f25635-fec1-46ac-9678-4ba245329c84-kube-api-access-6rw9v\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.990167 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.990145 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19415ce2-6c93-40de-9e76-02f21f6dd80b-config-volume\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.990513 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.990487 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19415ce2-6c93-40de-9e76-02f21f6dd80b-tmp-dir\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.994339 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.994316 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2f25635-fec1-46ac-9678-4ba245329c84-cert\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:32.998850 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.998825 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwvs\" (UniqueName: \"kubernetes.io/projected/19415ce2-6c93-40de-9e76-02f21f6dd80b-kube-api-access-4qwvs\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:32.998983 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:32.998891 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rw9v\" (UniqueName: \"kubernetes.io/projected/b2f25635-fec1-46ac-9678-4ba245329c84-kube-api-access-6rw9v\") pod \"ingress-canary-htv2f\" (UID: \"b2f25635-fec1-46ac-9678-4ba245329c84\") " pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:33.055031 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:33.054992 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-htv2f" Jul 21 08:19:33.494762 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:33.494718 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:33.494974 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:33.494899 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:33.494974 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:33.494965 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:19:34.494946051 +0000 UTC m=+37.158215254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:34.501654 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:34.501621 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:34.502140 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:34.501762 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:34.502140 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:34.501823 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:19:36.501806938 +0000 UTC m=+39.165076143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:34.663719 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:34.663523 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-htv2f"] Jul 21 08:19:34.724112 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:34.724067 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f25635_fec1_46ac_9678_4ba245329c84.slice/crio-172d4f9510ed6e5353e2d19c2c8b7e91b4a3fa47e1e3d9f1f536dca6e708d56b WatchSource:0}: Error finding container 172d4f9510ed6e5353e2d19c2c8b7e91b4a3fa47e1e3d9f1f536dca6e708d56b: Status 404 returned error can't find the container with id 172d4f9510ed6e5353e2d19c2c8b7e91b4a3fa47e1e3d9f1f536dca6e708d56b Jul 21 08:19:35.040617 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:35.040580 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htv2f" event={"ID":"b2f25635-fec1-46ac-9678-4ba245329c84","Type":"ContainerStarted","Data":"172d4f9510ed6e5353e2d19c2c8b7e91b4a3fa47e1e3d9f1f536dca6e708d56b"} Jul 21 08:19:35.043258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:35.043232 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="e1927e1eb337e8eeffd5d2d9699524380cd4ef33995c1ead2717e08f2ba34a8d" exitCode=0 Jul 21 08:19:35.043412 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:35.043289 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"e1927e1eb337e8eeffd5d2d9699524380cd4ef33995c1ead2717e08f2ba34a8d"} Jul 21 08:19:36.049998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:36.049961 2569 generic.go:356] "Generic (PLEG): container finished" podID="a8834bb4-ae86-4d20-b4d8-25f8cd803703" containerID="dcd09f85f00dcc18980266b6788c8cbf6ad68c330e4838ffb005c8544b53e478" exitCode=0 Jul 21 08:19:36.050656 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:36.050039 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerDied","Data":"dcd09f85f00dcc18980266b6788c8cbf6ad68c330e4838ffb005c8544b53e478"} Jul 21 08:19:36.517765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:36.517528 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:36.517951 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:36.517728 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:36.517951 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:36.517890 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:19:40.517868002 +0000 UTC m=+43.181137214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:37.060907 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:37.060866 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" event={"ID":"a8834bb4-ae86-4d20-b4d8-25f8cd803703","Type":"ContainerStarted","Data":"c96c2d34630f5fd4c3d1447307bbbdc2df6af04c3190a6ab5d65dce2cee37b1a"} Jul 21 08:19:37.083189 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:37.082488 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mqzlz" podStartSLOduration=9.46418587 podStartE2EDuration="40.082464116s" totalImagesPullingTime="30.618278246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:37.081193416 +0000 UTC m=+39.744462643" watchObservedRunningTime="2026-07-21 08:19:37.082464116 +0000 UTC m=+39.745733345" Jul 21 08:19:38.064721 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:38.064670 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-htv2f" event={"ID":"b2f25635-fec1-46ac-9678-4ba245329c84","Type":"ContainerStarted","Data":"aedf459cdaf99c91a33fdfd835a285846afeb3d2d6de9ac7387f58a07bb99824"} Jul 21 08:19:38.079270 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:38.079214 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-htv2f" podStartSLOduration=3.23193542 podStartE2EDuration="6.079190605s" totalImagesPullingTime="2.847255185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:19:38.078163259 +0000 UTC m=+40.741432484" watchObservedRunningTime="2026-07-21 08:19:38.079190605 +0000 UTC m=+40.742459832" Jul 21 08:19:40.546380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:40.546336 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:40.546819 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:40.546525 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:40.546819 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:40.546602 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:19:48.546583909 +0000 UTC m=+51.209853113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:48.600643 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:48.600597 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:19:48.601122 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:48.600737 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:19:48.601122 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:19:48.600804 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:20:04.600789272 +0000 UTC m=+67.264058474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:19:58.414732 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.414697 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k"] Jul 21 08:19:58.417369 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.417349 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.419728 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.419702 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 08:19:58.419908 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.419703 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 08:19:58.419986 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.419960 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 08:19:58.420887 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.420868 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hxn4b\"" Jul 21 08:19:58.420887 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.420886 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 21 08:19:58.426747 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.426725 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k"] Jul 21 08:19:58.477022 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.476986 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87dd\" (UniqueName: \"kubernetes.io/projected/cc3e9e93-7acb-4440-87a3-817d295aaffe-kube-api-access-p87dd\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.477206 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.477040 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc3e9e93-7acb-4440-87a3-817d295aaffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.577386 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.577357 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc3e9e93-7acb-4440-87a3-817d295aaffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.577484 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.577422 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87dd\" (UniqueName: \"kubernetes.io/projected/cc3e9e93-7acb-4440-87a3-817d295aaffe-kube-api-access-p87dd\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.581710 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.581668 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc3e9e93-7acb-4440-87a3-817d295aaffe-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.584481 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.584455 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87dd\" (UniqueName: \"kubernetes.io/projected/cc3e9e93-7acb-4440-87a3-817d295aaffe-kube-api-access-p87dd\") pod \"managed-serviceaccount-addon-agent-54b8c8c676-75f4k\" (UID: \"cc3e9e93-7acb-4440-87a3-817d295aaffe\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.735395 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.735294 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" Jul 21 08:19:58.862808 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:58.862770 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k"] Jul 21 08:19:58.865888 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:19:58.865859 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc3e9e93_7acb_4440_87a3_817d295aaffe.slice/crio-22d295fb87903cd6f9da4a14c9a962392b5bb8e0e9cdbc4391e53de388767483 WatchSource:0}: Error finding container 22d295fb87903cd6f9da4a14c9a962392b5bb8e0e9cdbc4391e53de388767483: Status 404 returned error can't find the container with id 22d295fb87903cd6f9da4a14c9a962392b5bb8e0e9cdbc4391e53de388767483 Jul 21 08:19:59.116907 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:19:59.116867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" event={"ID":"cc3e9e93-7acb-4440-87a3-817d295aaffe","Type":"ContainerStarted","Data":"22d295fb87903cd6f9da4a14c9a962392b5bb8e0e9cdbc4391e53de388767483"} Jul 21 08:20:00.039274 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:00.039245 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kt5wl" Jul 21 08:20:02.125647 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:02.125613 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" event={"ID":"cc3e9e93-7acb-4440-87a3-817d295aaffe","Type":"ContainerStarted","Data":"ddbc403319e1fa0d9139becc78c7bf96c62a6b42d6423e908de00faa2ccdca6c"} Jul 21 08:20:02.140635 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:02.140583 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" podStartSLOduration=1.842398061 podStartE2EDuration="4.140567164s" totalImagesPullingTime="2.298169103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:20:02.139850414 +0000 UTC m=+64.803119640" watchObservedRunningTime="2026-07-21 08:20:02.140567164 +0000 UTC m=+64.803836389" Jul 21 08:20:03.511934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.511887 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:20:03.512410 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.511979 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:20:03.514843 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.514822 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 08:20:03.514895 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.514829 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 08:20:03.522669 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:03.522627 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 08:20:03.522821 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:03.522748 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:21:07.522723635 +0000 UTC m=+130.185992838 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : secret "metrics-daemon-secret" not found Jul 21 08:20:03.524962 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.524943 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de0d6800-e8cd-4757-b2b7-9ee26630ec2a-original-pull-secret\") pod \"global-pull-secret-syncer-jkrrd\" (UID: \"de0d6800-e8cd-4757-b2b7-9ee26630ec2a\") " pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:20:03.613306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.613267 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:20:03.615950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.615929 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 08:20:03.625916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.625893 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 08:20:03.637308 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.637276 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7n5j\" (UniqueName: \"kubernetes.io/projected/92a57fce-2394-4a70-9686-a0f6ed680a28-kube-api-access-h7n5j\") pod \"network-check-target-vrc2v\" (UID: \"92a57fce-2394-4a70-9686-a0f6ed680a28\") " pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:20:03.740765 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.740714 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jkrrd" Jul 21 08:20:03.756326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.756299 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4qzvw\"" Jul 21 08:20:03.764245 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.764172 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:20:03.899844 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.899764 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jkrrd"] Jul 21 08:20:03.903309 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:20:03.903275 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde0d6800_e8cd_4757_b2b7_9ee26630ec2a.slice/crio-aade9a2a0d8476397b03b51a02c2e45d535b8ce9f500d75c62ca2cd9bde686dc WatchSource:0}: Error finding container aade9a2a0d8476397b03b51a02c2e45d535b8ce9f500d75c62ca2cd9bde686dc: Status 404 returned error can't find the container with id aade9a2a0d8476397b03b51a02c2e45d535b8ce9f500d75c62ca2cd9bde686dc Jul 21 08:20:03.916504 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:03.916473 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vrc2v"] Jul 21 08:20:03.919938 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:20:03.919916 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a57fce_2394_4a70_9686_a0f6ed680a28.slice/crio-18e8a7a22578cde6d179f089a76a44a104a118a10044041a163cf1ea06879d39 WatchSource:0}: Error finding container 18e8a7a22578cde6d179f089a76a44a104a118a10044041a163cf1ea06879d39: Status 404 returned error can't find the container with id 18e8a7a22578cde6d179f089a76a44a104a118a10044041a163cf1ea06879d39 Jul 21 08:20:04.130947 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:04.130847 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkrrd" event={"ID":"de0d6800-e8cd-4757-b2b7-9ee26630ec2a","Type":"ContainerStarted","Data":"aade9a2a0d8476397b03b51a02c2e45d535b8ce9f500d75c62ca2cd9bde686dc"} Jul 21 08:20:04.131930 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:04.131901 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrc2v" event={"ID":"92a57fce-2394-4a70-9686-a0f6ed680a28","Type":"ContainerStarted","Data":"18e8a7a22578cde6d179f089a76a44a104a118a10044041a163cf1ea06879d39"} Jul 21 08:20:04.624864 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:04.624340 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:20:04.624864 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:04.624490 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:20:04.624864 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:04.624549 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:20:36.624528983 +0000 UTC m=+99.287798234 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:20:09.147286 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:09.147166 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jkrrd" event={"ID":"de0d6800-e8cd-4757-b2b7-9ee26630ec2a","Type":"ContainerStarted","Data":"e3b7bf8a4b56c15d6fd9b50c8908a0be85502476ae688a7a3b47fb58074f2882"} Jul 21 08:20:09.148663 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:09.148638 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vrc2v" event={"ID":"92a57fce-2394-4a70-9686-a0f6ed680a28","Type":"ContainerStarted","Data":"994f3ee4ed7f78c11dc4a4d64a88070061a9d999c6124fa9f9746f2807478388"} Jul 21 08:20:09.148807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:09.148787 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:20:09.172803 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:09.172744 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jkrrd" podStartSLOduration=67.14686858499999 podStartE2EDuration="1m12.172723545s" totalImagesPullingTime="5.02585496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:20:09.159877953 +0000 UTC m=+71.823147179" watchObservedRunningTime="2026-07-21 08:20:09.172723545 +0000 UTC m=+71.835992774" Jul 21 08:20:09.173133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:09.173097 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vrc2v" podStartSLOduration=67.175296468 podStartE2EDuration="1m12.173085108s" totalImagesPullingTime="4.99778864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:20:09.172610563 +0000 UTC m=+71.835879788" watchObservedRunningTime="2026-07-21 08:20:09.173085108 +0000 UTC m=+71.836354335" Jul 21 08:20:36.642113 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:36.642064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:20:36.642544 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:36.642235 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:20:36.642544 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:20:36.642317 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:21:40.642297537 +0000 UTC m=+163.305566740 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:20:40.153184 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:20:40.153153 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vrc2v" Jul 21 08:21:07.557113 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:21:07.557064 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:21:07.557725 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:07.557206 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 08:21:07.557725 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:07.557288 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:23:09.557265048 +0000 UTC m=+252.220534256 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : secret "metrics-daemon-secret" not found Jul 21 08:21:35.748959 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:35.748909 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7qwx" podUID="19415ce2-6c93-40de-9e76-02f21f6dd80b" Jul 21 08:21:35.848240 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:35.848191 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:21:36.358568 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:21:36.358536 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:21:40.692784 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:21:40.692647 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:21:40.693196 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:40.692813 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 08:21:40.693196 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:21:40.692887 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls podName:19415ce2-6c93-40de-9e76-02f21f6dd80b nodeName:}" failed. No retries permitted until 2026-07-21 08:23:42.692869046 +0000 UTC m=+285.356138269 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls") pod "dns-default-l7qwx" (UID: "19415ce2-6c93-40de-9e76-02f21f6dd80b") : secret "dns-default-metrics-tls" not found Jul 21 08:21:47.829326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:21:47.829285 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:22:01.426405 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:22:01.426371 2569 generic.go:356] "Generic (PLEG): container finished" podID="cc3e9e93-7acb-4440-87a3-817d295aaffe" containerID="ddbc403319e1fa0d9139becc78c7bf96c62a6b42d6423e908de00faa2ccdca6c" exitCode=255 Jul 21 08:22:01.426829 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:22:01.426431 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" event={"ID":"cc3e9e93-7acb-4440-87a3-817d295aaffe","Type":"ContainerDied","Data":"ddbc403319e1fa0d9139becc78c7bf96c62a6b42d6423e908de00faa2ccdca6c"} Jul 21 08:22:01.432216 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:22:01.432197 2569 scope.go:117] "RemoveContainer" containerID="ddbc403319e1fa0d9139becc78c7bf96c62a6b42d6423e908de00faa2ccdca6c" Jul 21 08:22:02.435519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:22:02.435478 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54b8c8c676-75f4k" event={"ID":"cc3e9e93-7acb-4440-87a3-817d295aaffe","Type":"ContainerStarted","Data":"4da2f3f726b4681f4d6e4c8efb1021f3427aaca3e95ecf905f186055e9f1388a"} Jul 21 08:23:06.958611 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.958581 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:23:06.960545 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.960528 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp"] Jul 21 08:23:06.960664 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.960647 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:06.962471 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.962448 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:06.963736 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.963715 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 08:23:06.964048 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.964031 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 08:23:06.964116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.964036 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hvc2x\"" Jul 21 08:23:06.964116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.964027 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 08:23:06.965208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.965187 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 08:23:06.965324 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.965225 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 08:23:06.966691 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.966656 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 08:23:06.968262 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.968244 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-chfz2\"" Jul 21 08:23:06.974125 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.974017 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 08:23:06.974240 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.974217 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 08:23:06.977239 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.977216 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp"] Jul 21 08:23:06.982267 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:06.982242 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:23:07.036942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.036914 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l"] Jul 21 08:23:07.038935 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.038918 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8"] Jul 21 08:23:07.039055 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.039037 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" Jul 21 08:23:07.040761 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.040738 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2k5vl"] Jul 21 08:23:07.040870 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.040771 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" Jul 21 08:23:07.041926 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.041913 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 08:23:07.042173 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.042157 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-m5x9f\"" Jul 21 08:23:07.042258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.042242 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 08:23:07.042342 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.042327 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.042864 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.042845 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6grpk\"" Jul 21 08:23:07.044576 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.044555 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 08:23:07.045089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.045070 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 08:23:07.045173 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.045070 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j4phv\"" Jul 21 08:23:07.045365 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.045349 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 08:23:07.045736 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.045718 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 08:23:07.052288 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.052243 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 08:23:07.064092 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.064063 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8"] Jul 21 08:23:07.065016 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.064993 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l"] Jul 21 08:23:07.066523 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.066503 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2k5vl"] Jul 21 08:23:07.100279 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100238 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b9866716-ac4a-4838-b924-46cddc5d44f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.100279 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100276 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx2mx\" (UniqueName: \"kubernetes.io/projected/5542bb62-ebd1-458a-bfb8-f8d1df5bb431-kube-api-access-tx2mx\") pod \"volume-data-source-validator-78cf4c6f6-sjz2l\" (UID: \"5542bb62-ebd1-458a-bfb8-f8d1df5bb431\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100304 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-config\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100413 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100436 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knpgh\" (UniqueName: \"kubernetes.io/projected/f744f4f5-0076-4c02-800a-b60cfa0c389d-kube-api-access-knpgh\") pod \"network-check-source-7bc5b6bf85-nd8d8\" (UID: \"f744f4f5-0076-4c02-800a-b60cfa0c389d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100456 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100476 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9klrd\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100511 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100499 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb8b5\" (UniqueName: \"kubernetes.io/projected/bc47e7ac-e08a-464e-be52-f8791476c928-kube-api-access-vb8b5\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100522 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100539 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc47e7ac-e08a-464e-be52-f8791476c928-serving-cert\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100604 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100668 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-trusted-ca\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100714 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.100742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100738 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9r9\" (UniqueName: \"kubernetes.io/projected/b9866716-ac4a-4838-b924-46cddc5d44f0-kube-api-access-gn9r9\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.101081 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.101081 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.100804 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.201760 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201720 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201757 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201817 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-trusted-ca\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201847 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201871 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9r9\" (UniqueName: \"kubernetes.io/projected/b9866716-ac4a-4838-b924-46cddc5d44f0-kube-api-access-gn9r9\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201900 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.201958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201933 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201963 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b9866716-ac4a-4838-b924-46cddc5d44f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.201985 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx2mx\" (UniqueName: \"kubernetes.io/projected/5542bb62-ebd1-458a-bfb8-f8d1df5bb431-kube-api-access-tx2mx\") pod \"volume-data-source-validator-78cf4c6f6-sjz2l\" (UID: \"5542bb62-ebd1-458a-bfb8-f8d1df5bb431\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202011 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-config\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.202034 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.202057 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755ffdc496-n62ww: secret "image-registry-tls" not found Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202069 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202095 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knpgh\" (UniqueName: \"kubernetes.io/projected/f744f4f5-0076-4c02-800a-b60cfa0c389d-kube-api-access-knpgh\") pod \"network-check-source-7bc5b6bf85-nd8d8\" (UID: \"f744f4f5-0076-4c02-800a-b60cfa0c389d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202127 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.202155 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls podName:ebd0b393-e67a-4091-9c92-04171efdbc07 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:07.702133467 +0000 UTC m=+250.365402688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls") pod "image-registry-755ffdc496-n62ww" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07") : secret "image-registry-tls" not found Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.202194 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.202241 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:07.702226921 +0000 UTC m=+250.365496138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202194 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9klrd\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb8b5\" (UniqueName: \"kubernetes.io/projected/bc47e7ac-e08a-464e-be52-f8791476c928-kube-api-access-vb8b5\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202317 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202343 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc47e7ac-e08a-464e-be52-f8791476c928-serving-cert\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.203466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202541 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.204258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202838 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b9866716-ac4a-4838-b924-46cddc5d44f0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.204258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.202858 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.204258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.203002 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-trusted-ca\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.204258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.203036 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc47e7ac-e08a-464e-be52-f8791476c928-config\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.204258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.203774 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.204985 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.204962 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.205079 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.204964 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.205079 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.204968 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc47e7ac-e08a-464e-be52-f8791476c928-serving-cert\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.211771 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.211706 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb8b5\" (UniqueName: \"kubernetes.io/projected/bc47e7ac-e08a-464e-be52-f8791476c928-kube-api-access-vb8b5\") pod \"console-operator-565b7b569d-2k5vl\" (UID: \"bc47e7ac-e08a-464e-be52-f8791476c928\") " pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.215969 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.212312 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9r9\" (UniqueName: \"kubernetes.io/projected/b9866716-ac4a-4838-b924-46cddc5d44f0-kube-api-access-gn9r9\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.219711 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.219666 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9klrd\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.219829 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.219803 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.219882 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.219803 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx2mx\" (UniqueName: \"kubernetes.io/projected/5542bb62-ebd1-458a-bfb8-f8d1df5bb431-kube-api-access-tx2mx\") pod \"volume-data-source-validator-78cf4c6f6-sjz2l\" (UID: \"5542bb62-ebd1-458a-bfb8-f8d1df5bb431\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" Jul 21 08:23:07.220238 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.220222 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knpgh\" (UniqueName: \"kubernetes.io/projected/f744f4f5-0076-4c02-800a-b60cfa0c389d-kube-api-access-knpgh\") pod \"network-check-source-7bc5b6bf85-nd8d8\" (UID: \"f744f4f5-0076-4c02-800a-b60cfa0c389d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" Jul 21 08:23:07.349885 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.349832 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" Jul 21 08:23:07.355724 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.355704 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" Jul 21 08:23:07.359595 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.359569 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:07.501675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.501645 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l"] Jul 21 08:23:07.504802 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:07.504775 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5542bb62_ebd1_458a_bfb8_f8d1df5bb431.slice/crio-b45b9812f00128bcbc1a1a0df7d6df9de72e4978c037b97eb0138442033e6592 WatchSource:0}: Error finding container b45b9812f00128bcbc1a1a0df7d6df9de72e4978c037b97eb0138442033e6592: Status 404 returned error can't find the container with id b45b9812f00128bcbc1a1a0df7d6df9de72e4978c037b97eb0138442033e6592 Jul 21 08:23:07.625939 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.625904 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" event={"ID":"5542bb62-ebd1-458a-bfb8-f8d1df5bb431","Type":"ContainerStarted","Data":"b45b9812f00128bcbc1a1a0df7d6df9de72e4978c037b97eb0138442033e6592"} Jul 21 08:23:07.707241 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.707193 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:07.707423 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.707309 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:07.707423 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.707365 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:07.707526 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.707455 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 08:23:07.707526 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.707467 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:08.707443158 +0000 UTC m=+251.370712362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:07.707526 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.707471 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755ffdc496-n62ww: secret "image-registry-tls" not found Jul 21 08:23:07.707526 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:07.707521 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls podName:ebd0b393-e67a-4091-9c92-04171efdbc07 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:08.707504794 +0000 UTC m=+251.370774015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls") pod "image-registry-755ffdc496-n62ww" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07") : secret "image-registry-tls" not found Jul 21 08:23:07.722932 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.722911 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8"] Jul 21 08:23:07.725694 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:07.725651 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf744f4f5_0076_4c02_800a_b60cfa0c389d.slice/crio-0a37cf9d59711b8b0213c09555d946ecdd7f9d98371532b29f0f03819bef6442 WatchSource:0}: Error finding container 0a37cf9d59711b8b0213c09555d946ecdd7f9d98371532b29f0f03819bef6442: Status 404 returned error can't find the container with id 0a37cf9d59711b8b0213c09555d946ecdd7f9d98371532b29f0f03819bef6442 Jul 21 08:23:07.726373 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:07.726354 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2k5vl"] Jul 21 08:23:07.729061 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:07.729044 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc47e7ac_e08a_464e_be52_f8791476c928.slice/crio-3b8b8446d4b5c574afdb8f2191247b19e8d770d20a4f570f6e091c402abb255a WatchSource:0}: Error finding container 3b8b8446d4b5c574afdb8f2191247b19e8d770d20a4f570f6e091c402abb255a: Status 404 returned error can't find the container with id 3b8b8446d4b5c574afdb8f2191247b19e8d770d20a4f570f6e091c402abb255a Jul 21 08:23:08.632662 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:08.632626 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" event={"ID":"bc47e7ac-e08a-464e-be52-f8791476c928","Type":"ContainerStarted","Data":"3b8b8446d4b5c574afdb8f2191247b19e8d770d20a4f570f6e091c402abb255a"} Jul 21 08:23:08.634438 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:08.634407 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" event={"ID":"f744f4f5-0076-4c02-800a-b60cfa0c389d","Type":"ContainerStarted","Data":"b2ebb5fc35fe8cccd9c43cad496590cf0ff681b5e493352b811290b3a2efb728"} Jul 21 08:23:08.634588 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:08.634456 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" event={"ID":"f744f4f5-0076-4c02-800a-b60cfa0c389d","Type":"ContainerStarted","Data":"0a37cf9d59711b8b0213c09555d946ecdd7f9d98371532b29f0f03819bef6442"} Jul 21 08:23:08.715352 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:08.715310 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:08.715541 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:08.715409 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:08.715541 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:08.715514 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:08.715541 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:08.715513 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 08:23:08.715541 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:08.715534 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755ffdc496-n62ww: secret "image-registry-tls" not found Jul 21 08:23:08.715795 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:08.715573 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:10.715555346 +0000 UTC m=+253.378824548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:08.715795 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:08.715591 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls podName:ebd0b393-e67a-4091-9c92-04171efdbc07 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:10.715581699 +0000 UTC m=+253.378850902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls") pod "image-registry-755ffdc496-n62ww" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07") : secret "image-registry-tls" not found Jul 21 08:23:09.622535 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:09.622498 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:23:09.622764 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:09.622661 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 21 08:23:09.622764 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:09.622759 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs podName:d71ad86f-e6b6-45b0-87dc-63af669ddfff nodeName:}" failed. No retries permitted until 2026-07-21 08:25:11.622738903 +0000 UTC m=+374.286008150 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs") pod "network-metrics-daemon-zfdwm" (UID: "d71ad86f-e6b6-45b0-87dc-63af669ddfff") : secret "metrics-daemon-secret" not found Jul 21 08:23:09.639953 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:09.639916 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" event={"ID":"5542bb62-ebd1-458a-bfb8-f8d1df5bb431","Type":"ContainerStarted","Data":"a23a385104915e8337b2037dd8d750e293bfdeb31d22cb67953ec45f643f6316"} Jul 21 08:23:09.657167 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:09.657041 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-nd8d8" podStartSLOduration=2.657020404 podStartE2EDuration="2.657020404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:08.650660095 +0000 UTC m=+251.313929319" watchObservedRunningTime="2026-07-21 08:23:09.657020404 +0000 UTC m=+252.320289630" Jul 21 08:23:09.657891 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:09.657853 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-sjz2l" podStartSLOduration=1.380534416 podStartE2EDuration="2.657840723s" totalImagesPullingTime="1.277306307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:09.6568031 +0000 UTC m=+252.320072324" watchObservedRunningTime="2026-07-21 08:23:09.657840723 +0000 UTC m=+252.321109949" Jul 21 08:23:10.646194 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.646157 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/0.log" Jul 21 08:23:10.646578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.646212 2569 generic.go:356] "Generic (PLEG): container finished" podID="bc47e7ac-e08a-464e-be52-f8791476c928" containerID="b83920c9bd260f5f095fabccb4ed5aa004300a3b5d0562f1ddd1227f69bf2372" exitCode=255 Jul 21 08:23:10.646578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.646267 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" event={"ID":"bc47e7ac-e08a-464e-be52-f8791476c928","Type":"ContainerDied","Data":"b83920c9bd260f5f095fabccb4ed5aa004300a3b5d0562f1ddd1227f69bf2372"} Jul 21 08:23:10.646578 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.646532 2569 scope.go:117] "RemoveContainer" containerID="b83920c9bd260f5f095fabccb4ed5aa004300a3b5d0562f1ddd1227f69bf2372" Jul 21 08:23:10.730392 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.730352 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:10.730599 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:10.730473 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:10.730599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:10.730491 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:10.730599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:10.730563 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:14.730544457 +0000 UTC m=+257.393813665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:10.730599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:10.730573 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 08:23:10.730599 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:10.730585 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755ffdc496-n62ww: secret "image-registry-tls" not found Jul 21 08:23:10.730875 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:10.730637 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls podName:ebd0b393-e67a-4091-9c92-04171efdbc07 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:14.730620373 +0000 UTC m=+257.393889578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls") pod "image-registry-755ffdc496-n62ww" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07") : secret "image-registry-tls" not found Jul 21 08:23:11.633968 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.633932 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7thh8"] Jul 21 08:23:11.636205 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.636189 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.639397 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.639369 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 08:23:11.639591 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.639574 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 08:23:11.639699 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.639606 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 08:23:11.642050 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.642031 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 08:23:11.645907 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.645887 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gbchk\"" Jul 21 08:23:11.650353 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.650333 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/1.log" Jul 21 08:23:11.650743 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.650730 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/0.log" Jul 21 08:23:11.650783 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.650766 2569 generic.go:356] "Generic (PLEG): container finished" podID="bc47e7ac-e08a-464e-be52-f8791476c928" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" exitCode=255 Jul 21 08:23:11.650838 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.650822 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" event={"ID":"bc47e7ac-e08a-464e-be52-f8791476c928","Type":"ContainerDied","Data":"a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5"} Jul 21 08:23:11.650894 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.650881 2569 scope.go:117] "RemoveContainer" containerID="b83920c9bd260f5f095fabccb4ed5aa004300a3b5d0562f1ddd1227f69bf2372" Jul 21 08:23:11.651105 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.651090 2569 scope.go:117] "RemoveContainer" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" Jul 21 08:23:11.651296 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:11.651274 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:11.657176 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.657140 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7thh8"] Jul 21 08:23:11.738604 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.738556 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.738827 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.738653 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c147d75-3b2f-47f8-bd29-471adfa4f882-crio-socket\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.738827 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.738725 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c147d75-3b2f-47f8-bd29-471adfa4f882-data-volume\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.738919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.738856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgszj\" (UniqueName: \"kubernetes.io/projected/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-api-access-zgszj\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.738919 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.738902 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839464 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839425 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839464 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839466 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c147d75-3b2f-47f8-bd29-471adfa4f882-crio-socket\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839486 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c147d75-3b2f-47f8-bd29-471adfa4f882-data-volume\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgszj\" (UniqueName: \"kubernetes.io/projected/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-api-access-zgszj\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839530 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:11.839546 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839574 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c147d75-3b2f-47f8-bd29-471adfa4f882-crio-socket\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:11.839616 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:12.33960087 +0000 UTC m=+255.002870072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : configmap "kube-rbac-proxy" not found Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:11.839620 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:11.839666 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:11.839663 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:12.339649386 +0000 UTC m=+255.002918599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:11.839950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.839810 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c147d75-3b2f-47f8-bd29-471adfa4f882-data-volume\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:11.849544 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:11.849514 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgszj\" (UniqueName: \"kubernetes.io/projected/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-api-access-zgszj\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:12.343502 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:12.343444 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:12.343724 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:12.343537 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:12.343724 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:12.343583 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 08:23:12.343724 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:12.343660 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:13.343641703 +0000 UTC m=+256.006910907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : configmap "kube-rbac-proxy" not found Jul 21 08:23:12.343946 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:12.343722 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:12.343946 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:12.343791 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:13.343773842 +0000 UTC m=+256.007043062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:12.655320 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:12.655234 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/1.log" Jul 21 08:23:12.655780 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:12.655559 2569 scope.go:117] "RemoveContainer" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" Jul 21 08:23:12.655780 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:12.655727 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:13.352950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:13.352915 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:13.353132 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:13.352965 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:13.353132 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:13.353034 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 08:23:13.353132 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:13.353087 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:13.353132 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:13.353100 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:15.353083771 +0000 UTC m=+258.016352973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : configmap "kube-rbac-proxy" not found Jul 21 08:23:13.353132 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:13.353130 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:15.353119099 +0000 UTC m=+258.016388301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:13.399631 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:13.399600 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpg6q_4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4/dns-node-resolver/0.log" Jul 21 08:23:14.460125 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.460086 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vhmgg"] Jul 21 08:23:14.463019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.463002 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.465358 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.465334 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 21 08:23:14.465500 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.465410 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 21 08:23:14.465791 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.465773 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 21 08:23:14.466397 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.466382 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b4xxs\"" Jul 21 08:23:14.466471 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.466416 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 21 08:23:14.472701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.472662 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vhmgg"] Jul 21 08:23:14.564004 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.563963 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b24b19a3-d183-402c-ab87-874a761100ad-signing-key\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.564216 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.564020 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87pvv\" (UniqueName: \"kubernetes.io/projected/b24b19a3-d183-402c-ab87-874a761100ad-kube-api-access-87pvv\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.564216 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.564166 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b24b19a3-d183-402c-ab87-874a761100ad-signing-cabundle\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.665088 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.665052 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87pvv\" (UniqueName: \"kubernetes.io/projected/b24b19a3-d183-402c-ab87-874a761100ad-kube-api-access-87pvv\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.665300 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.665156 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b24b19a3-d183-402c-ab87-874a761100ad-signing-cabundle\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.665300 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.665209 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b24b19a3-d183-402c-ab87-874a761100ad-signing-key\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.665812 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.665788 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b24b19a3-d183-402c-ab87-874a761100ad-signing-cabundle\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.667734 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.667713 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b24b19a3-d183-402c-ab87-874a761100ad-signing-key\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.673986 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.673960 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87pvv\" (UniqueName: \"kubernetes.io/projected/b24b19a3-d183-402c-ab87-874a761100ad-kube-api-access-87pvv\") pod \"service-ca-58598695f7-vhmgg\" (UID: \"b24b19a3-d183-402c-ab87-874a761100ad\") " pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.766228 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.766190 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:14.766437 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.766310 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:14.766437 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:14.766343 2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 08:23:14.766437 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:14.766367 2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-755ffdc496-n62ww: secret "image-registry-tls" not found Jul 21 08:23:14.766437 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:14.766423 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls podName:ebd0b393-e67a-4091-9c92-04171efdbc07 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:22.766408057 +0000 UTC m=+265.429677259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls") pod "image-registry-755ffdc496-n62ww" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07") : secret "image-registry-tls" not found Jul 21 08:23:14.766437 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:14.766427 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:14.766629 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:14.766475 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:22.766461149 +0000 UTC m=+265.429730351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:14.772610 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.772589 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-vhmgg" Jul 21 08:23:14.799906 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.799882 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tmmt9_3e35a47c-9f4f-4988-abe0-c7575f829bea/node-ca/0.log" Jul 21 08:23:14.899192 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:14.899159 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-vhmgg"] Jul 21 08:23:14.902810 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:14.902782 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24b19a3_d183_402c_ab87_874a761100ad.slice/crio-c4c94468c83111f9381c7fc343f45cc2da41aeeac85720abc5e0ed118af3b7a5 WatchSource:0}: Error finding container c4c94468c83111f9381c7fc343f45cc2da41aeeac85720abc5e0ed118af3b7a5: Status 404 returned error can't find the container with id c4c94468c83111f9381c7fc343f45cc2da41aeeac85720abc5e0ed118af3b7a5 Jul 21 08:23:15.371191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:15.371151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:15.371375 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:15.371287 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:15.371375 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:15.371329 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:15.371496 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:15.371406 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:19.371383818 +0000 UTC m=+262.034653044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:15.371496 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:15.371409 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 08:23:15.371496 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:15.371468 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:19.371453907 +0000 UTC m=+262.034723109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : configmap "kube-rbac-proxy" not found Jul 21 08:23:15.400449 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:15.400401 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htv2f_b2f25635-fec1-46ac-9678-4ba245329c84/serve-healthcheck-canary/0.log" Jul 21 08:23:15.666140 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:15.666056 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vhmgg" event={"ID":"b24b19a3-d183-402c-ab87-874a761100ad","Type":"ContainerStarted","Data":"c4c94468c83111f9381c7fc343f45cc2da41aeeac85720abc5e0ed118af3b7a5"} Jul 21 08:23:17.360279 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:17.360233 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:17.360709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:17.360301 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:17.360709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:17.360611 2569 scope.go:117] "RemoveContainer" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" Jul 21 08:23:17.360802 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:17.360782 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:17.674579 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:17.674499 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-vhmgg" event={"ID":"b24b19a3-d183-402c-ab87-874a761100ad","Type":"ContainerStarted","Data":"228ccc7ff86d2f5c2bf587c8fd5fb9a036414e040662c81d45c50e7fce6a142b"} Jul 21 08:23:17.702045 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:17.701971 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-vhmgg" podStartSLOduration=1.776984997 podStartE2EDuration="3.701953423s" totalImagesPullingTime="1.924968426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:17.69993306 +0000 UTC m=+260.363202283" watchObservedRunningTime="2026-07-21 08:23:17.701953423 +0000 UTC m=+260.365222646" Jul 21 08:23:19.408020 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:19.407984 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:19.408020 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:19.408039 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:19.408483 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:19.408134 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:19.408483 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:19.408135 2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 08:23:19.408483 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:19.408188 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:27.408174095 +0000 UTC m=+270.071443297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:19.408483 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:19.408201 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:27.408195179 +0000 UTC m=+270.071464381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : configmap "kube-rbac-proxy" not found Jul 21 08:23:22.838323 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:22.838283 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:22.838715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:22.838389 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:22.838715 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:22.838489 2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:22.838715 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:22.838545 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls podName:b9866716-ac4a-4838-b924-46cddc5d44f0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:38.838529058 +0000 UTC m=+281.501798277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-n62bp" (UID: "b9866716-ac4a-4838-b924-46cddc5d44f0") : secret "cluster-monitoring-operator-tls" not found Jul 21 08:23:22.840819 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:22.840798 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"image-registry-755ffdc496-n62ww\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:22.871089 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:22.871064 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:22.999632 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:22.999604 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:23:23.002374 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:23.002348 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd0b393_e67a_4091_9c92_04171efdbc07.slice/crio-04ea8e977167817f063a3b73385519efae94a32c6ded28c1a2206107a49801cf WatchSource:0}: Error finding container 04ea8e977167817f063a3b73385519efae94a32c6ded28c1a2206107a49801cf: Status 404 returned error can't find the container with id 04ea8e977167817f063a3b73385519efae94a32c6ded28c1a2206107a49801cf Jul 21 08:23:23.693876 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:23.693840 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" event={"ID":"ebd0b393-e67a-4091-9c92-04171efdbc07","Type":"ContainerStarted","Data":"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71"} Jul 21 08:23:23.694069 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:23.693891 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" event={"ID":"ebd0b393-e67a-4091-9c92-04171efdbc07","Type":"ContainerStarted","Data":"04ea8e977167817f063a3b73385519efae94a32c6ded28c1a2206107a49801cf"} Jul 21 08:23:23.694069 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:23.693981 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:23.714675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:23.714625 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" podStartSLOduration=17.714604526 podStartE2EDuration="17.714604526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:23.713841041 +0000 UTC m=+266.377110264" watchObservedRunningTime="2026-07-21 08:23:23.714604526 +0000 UTC m=+266.377873750" Jul 21 08:23:27.474291 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:27.474250 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:27.474722 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:27.474352 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:27.474722 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:27.474409 2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 08:23:27.474722 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:27.474486 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls podName:6c147d75-3b2f-47f8-bd29-471adfa4f882 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:43.474468452 +0000 UTC m=+286.137737654 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7thh8" (UID: "6c147d75-3b2f-47f8-bd29-471adfa4f882") : secret "insights-runtime-extractor-tls" not found Jul 21 08:23:27.474910 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:27.474892 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c147d75-3b2f-47f8-bd29-471adfa4f882-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:29.828269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:29.828232 2569 scope.go:117] "RemoveContainer" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" Jul 21 08:23:30.717956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.717931 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:23:30.718301 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.718282 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/1.log" Jul 21 08:23:30.718402 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.718321 2569 generic.go:356] "Generic (PLEG): container finished" podID="bc47e7ac-e08a-464e-be52-f8791476c928" containerID="7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0" exitCode=255 Jul 21 08:23:30.718442 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.718399 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" event={"ID":"bc47e7ac-e08a-464e-be52-f8791476c928","Type":"ContainerDied","Data":"7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0"} Jul 21 08:23:30.718477 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.718457 2569 scope.go:117] "RemoveContainer" containerID="a2332b26098128272934cbb6b8957e9a3ef0631b71508339708dfb3081d026d5" Jul 21 08:23:30.718800 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:30.718783 2569 scope.go:117] "RemoveContainer" containerID="7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0" Jul 21 08:23:30.718965 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:30.718940 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:31.722953 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:31.722926 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:23:36.880700 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.880649 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:23:36.923132 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.923098 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b68679575-75d6q"] Jul 21 08:23:36.928574 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.928555 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.939700 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939656 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b68679575-75d6q"] Jul 21 08:23:36.939820 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939737 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-certificates\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.939820 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939774 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-installation-pull-secrets\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.939820 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939802 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-trusted-ca\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.939937 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-tls\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.939937 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939913 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e81384f-2d5d-4329-9194-49a40988c4b4-ca-trust-extracted\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.940005 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939952 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vn75\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-kube-api-access-8vn75\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.940044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.939999 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-image-registry-private-configuration\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:36.940080 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:36.940046 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-bound-sa-token\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041010 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.040977 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-certificates\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041021 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-installation-pull-secrets\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041043 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-trusted-ca\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041060 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-tls\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e81384f-2d5d-4329-9194-49a40988c4b4-ca-trust-extracted\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041229 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041111 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vn75\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-kube-api-access-8vn75\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041468 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041264 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-image-registry-private-configuration\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.041468 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.041339 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-bound-sa-token\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.042236 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.042209 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e81384f-2d5d-4329-9194-49a40988c4b4-ca-trust-extracted\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.042757 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.042728 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-certificates\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.042838 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.042811 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e81384f-2d5d-4329-9194-49a40988c4b4-trusted-ca\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.043620 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.043595 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-registry-tls\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.043743 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.043723 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-installation-pull-secrets\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.044187 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.044164 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e81384f-2d5d-4329-9194-49a40988c4b4-image-registry-private-configuration\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.049948 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.049927 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-bound-sa-token\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.050398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.050381 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vn75\" (UniqueName: \"kubernetes.io/projected/6e81384f-2d5d-4329-9194-49a40988c4b4-kube-api-access-8vn75\") pod \"image-registry-5b68679575-75d6q\" (UID: \"6e81384f-2d5d-4329-9194-49a40988c4b4\") " pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.237312 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.237213 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.360487 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.360446 2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:37.360626 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.360505 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:23:37.360855 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.360842 2569 scope.go:117] "RemoveContainer" containerID="7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0" Jul 21 08:23:37.361019 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:37.360998 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:37.367957 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.367932 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b68679575-75d6q"] Jul 21 08:23:37.370848 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:37.370818 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e81384f_2d5d_4329_9194_49a40988c4b4.slice/crio-8ac6bd1f4a8e9481fe2486c15ba8b8dc361f2775a2435b0b5ec9033650dc9d8c WatchSource:0}: Error finding container 8ac6bd1f4a8e9481fe2486c15ba8b8dc361f2775a2435b0b5ec9033650dc9d8c: Status 404 returned error can't find the container with id 8ac6bd1f4a8e9481fe2486c15ba8b8dc361f2775a2435b0b5ec9033650dc9d8c Jul 21 08:23:37.741990 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.741955 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b68679575-75d6q" event={"ID":"6e81384f-2d5d-4329-9194-49a40988c4b4","Type":"ContainerStarted","Data":"3d74680bb80476ff2ab5b240ae06b8141adfede373e7c08401b73bb079cc9431"} Jul 21 08:23:37.742138 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.742014 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b68679575-75d6q" event={"ID":"6e81384f-2d5d-4329-9194-49a40988c4b4","Type":"ContainerStarted","Data":"8ac6bd1f4a8e9481fe2486c15ba8b8dc361f2775a2435b0b5ec9033650dc9d8c"} Jul 21 08:23:37.742138 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.742066 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:37.759960 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:37.759900 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b68679575-75d6q" podStartSLOduration=1.7598835990000001 podStartE2EDuration="1.759883599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:37.758797568 +0000 UTC m=+280.422066792" watchObservedRunningTime="2026-07-21 08:23:37.759883599 +0000 UTC m=+280.423152823" Jul 21 08:23:38.856633 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:38.856587 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:38.859101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:38.859079 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b9866716-ac4a-4838-b924-46cddc5d44f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-n62bp\" (UID: \"b9866716-ac4a-4838-b924-46cddc5d44f0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:39.077023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:39.076988 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" Jul 21 08:23:39.221969 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:39.221898 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp"] Jul 21 08:23:39.224586 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:39.224559 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9866716_ac4a_4838_b924_46cddc5d44f0.slice/crio-c86fc41d5df4612a6da78f7afb3f654c8b326b69f4c33947b4b48a15bc9f1ad5 WatchSource:0}: Error finding container c86fc41d5df4612a6da78f7afb3f654c8b326b69f4c33947b4b48a15bc9f1ad5: Status 404 returned error can't find the container with id c86fc41d5df4612a6da78f7afb3f654c8b326b69f4c33947b4b48a15bc9f1ad5 Jul 21 08:23:39.359602 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:39.359498 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7qwx" podUID="19415ce2-6c93-40de-9e76-02f21f6dd80b" Jul 21 08:23:39.749456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:39.749419 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" event={"ID":"b9866716-ac4a-4838-b924-46cddc5d44f0","Type":"ContainerStarted","Data":"c86fc41d5df4612a6da78f7afb3f654c8b326b69f4c33947b4b48a15bc9f1ad5"} Jul 21 08:23:39.749456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:39.749447 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:41.756659 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:41.756619 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" event={"ID":"b9866716-ac4a-4838-b924-46cddc5d44f0","Type":"ContainerStarted","Data":"0675d3eb4616ea6ba458f712f7766175ccbbbaae20fa1efe6e27cc901901b0dc"} Jul 21 08:23:41.778053 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:41.777999 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-n62bp" podStartSLOduration=33.805397024 podStartE2EDuration="35.777977946s" totalImagesPullingTime="1.972580922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:41.777207168 +0000 UTC m=+284.440476391" watchObservedRunningTime="2026-07-21 08:23:41.777977946 +0000 UTC m=+284.441247164" Jul 21 08:23:42.790108 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:42.790076 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:42.792391 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:42.792371 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19415ce2-6c93-40de-9e76-02f21f6dd80b-metrics-tls\") pod \"dns-default-l7qwx\" (UID: \"19415ce2-6c93-40de-9e76-02f21f6dd80b\") " pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:43.052675 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.052604 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jw7kl\"" Jul 21 08:23:43.060561 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.060541 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:43.183995 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.183961 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7qwx"] Jul 21 08:23:43.186759 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:43.186731 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19415ce2_6c93_40de_9e76_02f21f6dd80b.slice/crio-bd47f1bca8974a0ff5bf574795ad71f83d0989b3eb69dc85ff823f84dff1ff21 WatchSource:0}: Error finding container bd47f1bca8974a0ff5bf574795ad71f83d0989b3eb69dc85ff823f84dff1ff21: Status 404 returned error can't find the container with id bd47f1bca8974a0ff5bf574795ad71f83d0989b3eb69dc85ff823f84dff1ff21 Jul 21 08:23:43.497113 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.497088 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:43.499434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.499413 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c147d75-3b2f-47f8-bd29-471adfa4f882-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7thh8\" (UID: \"6c147d75-3b2f-47f8-bd29-471adfa4f882\") " pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:43.745044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.745006 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7thh8" Jul 21 08:23:43.763515 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.763488 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7qwx" event={"ID":"19415ce2-6c93-40de-9e76-02f21f6dd80b","Type":"ContainerStarted","Data":"bd47f1bca8974a0ff5bf574795ad71f83d0989b3eb69dc85ff823f84dff1ff21"} Jul 21 08:23:43.915542 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:43.915401 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7thh8"] Jul 21 08:23:43.918622 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:43.918589 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c147d75_3b2f_47f8_bd29_471adfa4f882.slice/crio-c3d8616afcb133179e96c8cd116ec24b12b4a38054c812694a95d3b1b35e8a60 WatchSource:0}: Error finding container c3d8616afcb133179e96c8cd116ec24b12b4a38054c812694a95d3b1b35e8a60: Status 404 returned error can't find the container with id c3d8616afcb133179e96c8cd116ec24b12b4a38054c812694a95d3b1b35e8a60 Jul 21 08:23:44.768311 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:44.768274 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7thh8" event={"ID":"6c147d75-3b2f-47f8-bd29-471adfa4f882","Type":"ContainerStarted","Data":"c3d8616afcb133179e96c8cd116ec24b12b4a38054c812694a95d3b1b35e8a60"} Jul 21 08:23:45.773891 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:45.773848 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7qwx" event={"ID":"19415ce2-6c93-40de-9e76-02f21f6dd80b","Type":"ContainerStarted","Data":"2ea401e9da8c893422ab01473ac03f6432b8a8c504b4c94cd64c9b9faf69399c"} Jul 21 08:23:45.774358 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:45.773914 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7qwx" event={"ID":"19415ce2-6c93-40de-9e76-02f21f6dd80b","Type":"ContainerStarted","Data":"210a0f41f7be2cb7676bd06bf7d578221dd38351d2e453059f33283b40ee0256"} Jul 21 08:23:45.774358 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:45.773990 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:45.792459 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:45.792408 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l7qwx" podStartSLOduration=252.043372355 podStartE2EDuration="4m13.792390678s" totalImagesPullingTime="1.749018323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:45.789831306 +0000 UTC m=+288.453100600" watchObservedRunningTime="2026-07-21 08:23:45.792390678 +0000 UTC m=+288.455659900" Jul 21 08:23:46.888246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:46.888210 2569 patch_prober.go:28] interesting pod/image-registry-755ffdc496-n62ww container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 21 08:23:46.888614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:46.888274 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 08:23:49.789389 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:49.789353 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7thh8" event={"ID":"6c147d75-3b2f-47f8-bd29-471adfa4f882","Type":"ContainerStarted","Data":"f612877d27b78ad63689df16ccd6020944d5c39581ae48bfcb6034798f0d217b"} Jul 21 08:23:49.829425 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:49.829393 2569 scope.go:117] "RemoveContainer" containerID="7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0" Jul 21 08:23:49.829720 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:49.829664 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-2k5vl_openshift-console-operator(bc47e7ac-e08a-464e-be52-f8791476c928)\"" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podUID="bc47e7ac-e08a-464e-be52-f8791476c928" Jul 21 08:23:50.829797 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:50.829754 2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-zfdwm" podUID="d71ad86f-e6b6-45b0-87dc-63af669ddfff" Jul 21 08:23:52.111694 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.111656 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl"] Jul 21 08:23:52.124098 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.124056 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.129998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.129478 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kr2l5\"" Jul 21 08:23:52.129998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.129824 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 21 08:23:52.129998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.129903 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl"] Jul 21 08:23:52.130504 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.130376 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 08:23:52.132711 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.132095 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mjtpp"] Jul 21 08:23:52.134443 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.134252 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 21 08:23:52.137437 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.137302 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.140862 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.140588 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xz96j"] Jul 21 08:23:52.143472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.142777 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7p8jk\"" Jul 21 08:23:52.143472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.142969 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 21 08:23:52.143472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.143150 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 21 08:23:52.143472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.143324 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 21 08:23:52.146113 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.146096 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.148894 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.148760 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 08:23:52.148894 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.148783 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x4bjv\"" Jul 21 08:23:52.149059 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.148915 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 08:23:52.149059 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.148922 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 08:23:52.155338 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.155252 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mjtpp"] Jul 21 08:23:52.181517 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.181481 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgwrb\" (UniqueName: \"kubernetes.io/projected/eda7ae38-e803-48a8-a915-8f09489ad530-kube-api-access-dgwrb\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.181730 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.181537 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eda7ae38-e803-48a8-a915-8f09489ad530-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.181730 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.181588 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.181730 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.181623 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.282457 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282412 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgwrb\" (UniqueName: \"kubernetes.io/projected/eda7ae38-e803-48a8-a915-8f09489ad530-kube-api-access-dgwrb\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282473 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282536 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eda7ae38-e803-48a8-a915-8f09489ad530-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282563 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-root\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282590 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282615 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.282660 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgtbn\" (UniqueName: \"kubernetes.io/projected/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-api-access-jgtbn\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282703 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-metrics-client-ca\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282732 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282765 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282788 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-wtmp\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282818 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-textfile\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282857 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm8zj\" (UniqueName: \"kubernetes.io/projected/e206cc46-0d54-4324-b528-85142c32ee87-kube-api-access-nm8zj\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282885 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282917 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282947 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-sys\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.282969 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.283023 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.283570 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.283364 2569 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 21 08:23:52.283570 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.283443 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls podName:eda7ae38-e803-48a8-a915-8f09489ad530 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:52.783421882 +0000 UTC m=+295.446691090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-zq7tl" (UID: "eda7ae38-e803-48a8-a915-8f09489ad530") : secret "openshift-state-metrics-tls" not found Jul 21 08:23:52.283978 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.283911 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eda7ae38-e803-48a8-a915-8f09489ad530-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.286360 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.286327 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.295623 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.295593 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgwrb\" (UniqueName: \"kubernetes.io/projected/eda7ae38-e803-48a8-a915-8f09489ad530-kube-api-access-dgwrb\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.384306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384216 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgtbn\" (UniqueName: \"kubernetes.io/projected/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-api-access-jgtbn\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.384306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384265 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-metrics-client-ca\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384296 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384325 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384348 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-wtmp\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-textfile\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384405 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm8zj\" (UniqueName: \"kubernetes.io/projected/e206cc46-0d54-4324-b528-85142c32ee87-kube-api-access-nm8zj\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384482 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-sys\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384504 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.384577 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384547 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.385039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384583 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.385039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384608 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.385039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384648 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-root\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.385039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384826 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-root\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.385039 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.384905 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-textfile\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.385277 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385258 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.385377 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385349 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385536 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385701 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-wtmp\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385776 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-metrics-client-ca\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.385838 2569 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.385907 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls podName:bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:52.88588848 +0000 UTC m=+295.549157682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-mjtpp" (UID: "bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43") : secret "kube-state-metrics-tls" not found Jul 21 08:23:52.386107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.385839 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e206cc46-0d54-4324-b528-85142c32ee87-sys\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.386481 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.386156 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 08:23:52.386481 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:52.386208 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls podName:e206cc46-0d54-4324-b528-85142c32ee87 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:52.8861929 +0000 UTC m=+295.549462108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls") pod "node-exporter-xz96j" (UID: "e206cc46-0d54-4324-b528-85142c32ee87") : secret "node-exporter-tls" not found Jul 21 08:23:52.386481 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.386252 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.389032 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.389012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.389177 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.389009 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.403383 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.403314 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgtbn\" (UniqueName: \"kubernetes.io/projected/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-api-access-jgtbn\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.405360 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.405326 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm8zj\" (UniqueName: \"kubernetes.io/projected/e206cc46-0d54-4324-b528-85142c32ee87-kube-api-access-nm8zj\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.788905 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.788865 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.791872 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.791832 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda7ae38-e803-48a8-a915-8f09489ad530-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-zq7tl\" (UID: \"eda7ae38-e803-48a8-a915-8f09489ad530\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:52.889818 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.889788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.889983 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.889874 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:52.892390 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.892364 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-mjtpp\" (UID: \"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:52.892508 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:52.892373 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e206cc46-0d54-4324-b528-85142c32ee87-node-exporter-tls\") pod \"node-exporter-xz96j\" (UID: \"e206cc46-0d54-4324-b528-85142c32ee87\") " pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:53.042192 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.042101 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" Jul 21 08:23:53.054133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.054104 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" Jul 21 08:23:53.062310 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.062278 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xz96j" Jul 21 08:23:53.074170 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:53.074139 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode206cc46_0d54_4324_b528_85142c32ee87.slice/crio-9282977a79a75a04e8545baa8a0bd85f49cce33fe8291d030a36e32599048c6b WatchSource:0}: Error finding container 9282977a79a75a04e8545baa8a0bd85f49cce33fe8291d030a36e32599048c6b: Status 404 returned error can't find the container with id 9282977a79a75a04e8545baa8a0bd85f49cce33fe8291d030a36e32599048c6b Jul 21 08:23:53.204624 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.204589 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl"] Jul 21 08:23:53.207027 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:53.207000 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda7ae38_e803_48a8_a915_8f09489ad530.slice/crio-bbea3c2bda4613fb87f065c14af17d03bcf3ba0c4b495a8a93115d31d9a1a1e9 WatchSource:0}: Error finding container bbea3c2bda4613fb87f065c14af17d03bcf3ba0c4b495a8a93115d31d9a1a1e9: Status 404 returned error can't find the container with id bbea3c2bda4613fb87f065c14af17d03bcf3ba0c4b495a8a93115d31d9a1a1e9 Jul 21 08:23:53.222998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.222969 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-mjtpp"] Jul 21 08:23:53.225718 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:53.225675 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbcbc716_3fcc_40c0_acda_2d3dd6f3bb43.slice/crio-dd221b72cd32acb6ed031c728451188c0bba70ef99f6627edd5a8dda68088ac9 WatchSource:0}: Error finding container dd221b72cd32acb6ed031c728451188c0bba70ef99f6627edd5a8dda68088ac9: Status 404 returned error can't find the container with id dd221b72cd32acb6ed031c728451188c0bba70ef99f6627edd5a8dda68088ac9 Jul 21 08:23:53.318054 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.317238 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:23:53.328075 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.328042 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.330997 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.330958 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 08:23:53.330997 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.330991 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 08:23:53.331182 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.330958 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 08:23:53.331182 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.331120 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 08:23:53.331182 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.330990 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 08:23:53.331987 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.331549 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zgctx\"" Jul 21 08:23:53.331987 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.331769 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 08:23:53.331987 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.331943 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 08:23:53.332199 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.332037 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 08:23:53.332199 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.332088 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 08:23:53.333036 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.333013 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394411 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394472 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394496 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394522 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394551 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394602 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394642 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.394701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394670 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.395051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394753 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.395051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394782 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.395051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394809 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.395051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394865 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.395051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.394906 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495373 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495334 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495373 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495372 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495548 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495595 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495751 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495628 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495751 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495663 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495751 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495731 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495882 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495779 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495882 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495814 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.495882 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495832 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.496018 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.495837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.496066 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.496014 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.496066 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.496053 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.496160 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.496084 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.497130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.497049 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.497130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.497079 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.497359 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:53.497338 2569 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 21 08:23:53.497433 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:23:53.497416 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls podName:632a76c5-52f3-42da-bf23-3928fef288b0 nodeName:}" failed. No retries permitted until 2026-07-21 08:23:53.997397095 +0000 UTC m=+296.660666302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0") : secret "alertmanager-main-tls" not found Jul 21 08:23:53.500058 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.500012 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.500214 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.500196 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.501590 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.501532 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.501727 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.501617 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.502012 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.501984 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.502099 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.502018 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.502408 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.502385 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.503890 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.503867 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.512400 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.512369 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:53.808010 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.807969 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7thh8" event={"ID":"6c147d75-3b2f-47f8-bd29-471adfa4f882","Type":"ContainerStarted","Data":"d521487ed6d6b3962d262ed4dc32e8c922825cd187facdf920149a054036e9f3"} Jul 21 08:23:53.810272 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.810239 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" event={"ID":"eda7ae38-e803-48a8-a915-8f09489ad530","Type":"ContainerStarted","Data":"c486573d95aaafadaa1439824a2e4715d4a5a2b9fb68316cd2c78bef45c9e0ab"} Jul 21 08:23:53.810406 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.810306 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" event={"ID":"eda7ae38-e803-48a8-a915-8f09489ad530","Type":"ContainerStarted","Data":"185018e4254c8fb037d4c86d86ded94efb3feb6a9498212b6808b513b7216180"} Jul 21 08:23:53.810406 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.810334 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" event={"ID":"eda7ae38-e803-48a8-a915-8f09489ad530","Type":"ContainerStarted","Data":"bbea3c2bda4613fb87f065c14af17d03bcf3ba0c4b495a8a93115d31d9a1a1e9"} Jul 21 08:23:53.811944 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.811913 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz96j" event={"ID":"e206cc46-0d54-4324-b528-85142c32ee87","Type":"ContainerStarted","Data":"9282977a79a75a04e8545baa8a0bd85f49cce33fe8291d030a36e32599048c6b"} Jul 21 08:23:53.813409 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:53.813387 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" event={"ID":"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43","Type":"ContainerStarted","Data":"dd221b72cd32acb6ed031c728451188c0bba70ef99f6627edd5a8dda68088ac9"} Jul 21 08:23:54.001112 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.001075 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:54.005576 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.005506 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:54.255462 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.255425 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:23:54.446729 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.446702 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:23:54.449742 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:23:54.449710 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632a76c5_52f3_42da_bf23_3928fef288b0.slice/crio-be80c082cb6e9bbb3cd4a086f35ad46b684c3258e68fa1193919f7614b89dd1d WatchSource:0}: Error finding container be80c082cb6e9bbb3cd4a086f35ad46b684c3258e68fa1193919f7614b89dd1d: Status 404 returned error can't find the container with id be80c082cb6e9bbb3cd4a086f35ad46b684c3258e68fa1193919f7614b89dd1d Jul 21 08:23:54.820732 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.820665 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"be80c082cb6e9bbb3cd4a086f35ad46b684c3258e68fa1193919f7614b89dd1d"} Jul 21 08:23:54.822788 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.822747 2569 generic.go:356] "Generic (PLEG): container finished" podID="e206cc46-0d54-4324-b528-85142c32ee87" containerID="10b1f70522f0b91b9cd6865ee04d1172fd43cff6da8ea31690bdd2aa9ac9970f" exitCode=0 Jul 21 08:23:54.822926 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:54.822806 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz96j" event={"ID":"e206cc46-0d54-4324-b528-85142c32ee87","Type":"ContainerDied","Data":"10b1f70522f0b91b9cd6865ee04d1172fd43cff6da8ea31690bdd2aa9ac9970f"} Jul 21 08:23:55.780444 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:55.780400 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l7qwx" Jul 21 08:23:56.887074 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:56.887046 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:23:57.242520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.242414 2569 patch_prober.go:28] interesting pod/image-registry-5b68679575-75d6q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 21 08:23:57.242520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.242485 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5b68679575-75d6q" podUID="6e81384f-2d5d-4329-9194-49a40988c4b4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 08:23:57.717656 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.717619 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:23:57.718617 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.718592 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:23:57.727539 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.727517 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:23:57.728130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.728107 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:23:57.731132 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.731102 2569 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 08:23:57.840949 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.840719 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7thh8" event={"ID":"6c147d75-3b2f-47f8-bd29-471adfa4f882","Type":"ContainerStarted","Data":"61439819c145090df9c22c903e4ddb09404c282676dda6086a44f7e4a18fce89"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.842932 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" event={"ID":"eda7ae38-e803-48a8-a915-8f09489ad530","Type":"ContainerStarted","Data":"477463bc8d7a9839a0fef2d890ba5a38f4bb927a9aba5256d90e34b7cf2dc4d3"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.844478 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767" exitCode=0 Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.844533 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.847260 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz96j" event={"ID":"e206cc46-0d54-4324-b528-85142c32ee87","Type":"ContainerStarted","Data":"87bd5dd635a9d7e0679e566b66ec31618ff633efd805780b9af7745932767ae6"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.847322 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz96j" event={"ID":"e206cc46-0d54-4324-b528-85142c32ee87","Type":"ContainerStarted","Data":"cb4ac8526c8a9b40f0b0e21df59b4e29e7422e1b881b9fd46c02870452296349"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.850922 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" event={"ID":"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43","Type":"ContainerStarted","Data":"dabfa5bcdb0f95823062bb6e2569737ef60262de342e6eb441f289dbc0db859e"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.850967 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" event={"ID":"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43","Type":"ContainerStarted","Data":"88de74673232607d6a78b8ef6d6c1e11c5804654a896e9f36705668ad00d66a4"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.850988 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" event={"ID":"bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43","Type":"ContainerStarted","Data":"2ae5bd46c808d271c49516b4f25bb3eb935ffc7eac6e785a19a8d6d0787fbd72"} Jul 21 08:23:57.865749 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.861149 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7thh8" podStartSLOduration=33.608251453 podStartE2EDuration="46.86110536s" totalImagesPullingTime="13.252853907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 08:23:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:57.860119637 +0000 UTC m=+300.523388860" watchObservedRunningTime="2026-07-21 08:23:57.86110536 +0000 UTC m=+300.524374583" Jul 21 08:23:57.879551 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.879495 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-mjtpp" podStartSLOduration=1.794443051 podStartE2EDuration="5.879475842s" totalImagesPullingTime="4.085032791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:57.878236591 +0000 UTC m=+300.541505814" watchObservedRunningTime="2026-07-21 08:23:57.879475842 +0000 UTC m=+300.542745076" Jul 21 08:23:57.897608 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.897545 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xz96j" podStartSLOduration=4.8147166420000005 podStartE2EDuration="5.897526678s" totalImagesPullingTime="988.908835ms" totalInitContainerRuntime="93.901201ms" isStatefulPod=true podCreationTimestamp="2026-07-21 08:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:57.896056566 +0000 UTC m=+300.559325791" watchObservedRunningTime="2026-07-21 08:23:57.897526678 +0000 UTC m=+300.560795892" Jul 21 08:23:57.939790 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:57.939731 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-zq7tl" podStartSLOduration=1.999788766 podStartE2EDuration="5.939711265s" totalImagesPullingTime="3.939922499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:23:57.938887601 +0000 UTC m=+300.602156828" watchObservedRunningTime="2026-07-21 08:23:57.939711265 +0000 UTC m=+300.602980486" Jul 21 08:23:58.752224 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:58.752190 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b68679575-75d6q" Jul 21 08:23:59.711879 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.711854 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:23:59.860969 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.860936 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be"} Jul 21 08:23:59.861146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.861004 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1"} Jul 21 08:23:59.861146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.861031 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94"} Jul 21 08:23:59.861146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.861054 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad"} Jul 21 08:23:59.861146 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:23:59.861075 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9"} Jul 21 08:24:00.869934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:00.869847 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerStarted","Data":"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb"} Jul 21 08:24:00.897389 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:00.897339 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.559696823 podStartE2EDuration="7.89732449s" totalImagesPullingTime="5.314979696s" totalInitContainerRuntime="22.647971ms" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:24:00.896089425 +0000 UTC m=+303.559358648" watchObservedRunningTime="2026-07-21 08:24:00.89732449 +0000 UTC m=+303.560593713" Jul 21 08:24:01.900272 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:01.900235 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerName="registry" containerID="cri-o://ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71" gracePeriod=30 Jul 21 08:24:02.142265 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.142240 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:24:02.277761 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277728 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277807 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277826 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277854 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277869 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277888 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.277921 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9klrd\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.278181 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.277944 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates\") pod \"ebd0b393-e67a-4091-9c92-04171efdbc07\" (UID: \"ebd0b393-e67a-4091-9c92-04171efdbc07\") " Jul 21 08:24:02.278528 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.278367 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:24:02.278847 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.278639 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:24:02.280512 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.280484 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:24:02.280613 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.280557 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:24:02.280613 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.280577 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd" (OuterVolumeSpecName: "kube-api-access-9klrd") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "kube-api-access-9klrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:24:02.280613 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.280559 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:24:02.280755 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.280651 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:24:02.286859 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.286834 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ebd0b393-e67a-4091-9c92-04171efdbc07" (UID: "ebd0b393-e67a-4091-9c92-04171efdbc07"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 08:24:02.378972 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.378936 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-image-registry-private-configuration\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.378972 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.378967 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-trusted-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.378972 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.378977 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebd0b393-e67a-4091-9c92-04171efdbc07-ca-trust-extracted\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.379174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.378984 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.379174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.378995 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebd0b393-e67a-4091-9c92-04171efdbc07-installation-pull-secrets\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.379174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.379005 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9klrd\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-kube-api-access-9klrd\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.379174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.379013 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebd0b393-e67a-4091-9c92-04171efdbc07-registry-certificates\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.379174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.379022 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebd0b393-e67a-4091-9c92-04171efdbc07-bound-sa-token\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:24:02.828307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.828280 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:24:02.878562 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.878531 2569 generic.go:356] "Generic (PLEG): container finished" podID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerID="ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71" exitCode=0 Jul 21 08:24:02.878763 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.878577 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" Jul 21 08:24:02.878763 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.878599 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" event={"ID":"ebd0b393-e67a-4091-9c92-04171efdbc07","Type":"ContainerDied","Data":"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71"} Jul 21 08:24:02.878763 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.878640 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755ffdc496-n62ww" event={"ID":"ebd0b393-e67a-4091-9c92-04171efdbc07","Type":"ContainerDied","Data":"04ea8e977167817f063a3b73385519efae94a32c6ded28c1a2206107a49801cf"} Jul 21 08:24:02.878763 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.878658 2569 scope.go:117] "RemoveContainer" containerID="ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71" Jul 21 08:24:02.887239 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.887221 2569 scope.go:117] "RemoveContainer" containerID="ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71" Jul 21 08:24:02.887546 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:24:02.887528 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71\": container with ID starting with ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71 not found: ID does not exist" containerID="ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71" Jul 21 08:24:02.887593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.887556 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71"} err="failed to get container status \"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71\": rpc error: code = NotFound desc = could not find container \"ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71\": container with ID starting with ba354ded833c6616682a68cad410c50441bd3ef7a09b5770436b0e9611f5bd71 not found: ID does not exist" Jul 21 08:24:02.898503 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.898470 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:24:02.901781 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:02.901757 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-755ffdc496-n62ww"] Jul 21 08:24:03.828651 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:03.828614 2569 scope.go:117] "RemoveContainer" containerID="7f23ab22b18510a8cc7aac47e58800d5810ba33d1896e32c1f064cee53fc48f0" Jul 21 08:24:03.832977 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:03.832952 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" path="/var/lib/kubelet/pods/ebd0b393-e67a-4091-9c92-04171efdbc07/volumes" Jul 21 08:24:03.885176 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:03.885151 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:24:04.892521 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:04.892495 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:24:04.892943 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:04.892572 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" event={"ID":"bc47e7ac-e08a-464e-be52-f8791476c928","Type":"ContainerStarted","Data":"ec4ced5e0cc6b07602d251d6ef41582b514bdf2b07e528f62c638ca82c940fea"} Jul 21 08:24:04.892943 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:04.892891 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:24:04.908154 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:04.908105 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" podStartSLOduration=55.617474912 podStartE2EDuration="57.908088456s" totalImagesPullingTime="2.290613544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:24:04.907599375 +0000 UTC m=+307.570868601" watchObservedRunningTime="2026-07-21 08:24:04.908088456 +0000 UTC m=+307.571357680" Jul 21 08:24:04.915952 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:04.915916 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-2k5vl" Jul 21 08:24:05.085285 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.085252 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-c7vr5"] Jul 21 08:24:05.085557 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.085545 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerName="registry" Jul 21 08:24:05.085627 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.085558 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerName="registry" Jul 21 08:24:05.085627 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.085620 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd0b393-e67a-4091-9c92-04171efdbc07" containerName="registry" Jul 21 08:24:05.088595 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.088572 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:05.090993 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.090970 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 08:24:05.091107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.090976 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 08:24:05.091107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.091070 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wsdjv\"" Jul 21 08:24:05.097445 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.097419 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-c7vr5"] Jul 21 08:24:05.207398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.207295 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-826dj\" (UniqueName: \"kubernetes.io/projected/397f1417-0b0a-4d74-94c1-704f3b33eee5-kube-api-access-826dj\") pod \"downloads-5bc67dcd46-c7vr5\" (UID: \"397f1417-0b0a-4d74-94c1-704f3b33eee5\") " pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:05.308203 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.308160 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-826dj\" (UniqueName: \"kubernetes.io/projected/397f1417-0b0a-4d74-94c1-704f3b33eee5-kube-api-access-826dj\") pod \"downloads-5bc67dcd46-c7vr5\" (UID: \"397f1417-0b0a-4d74-94c1-704f3b33eee5\") " pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:05.317380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.317347 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-826dj\" (UniqueName: \"kubernetes.io/projected/397f1417-0b0a-4d74-94c1-704f3b33eee5-kube-api-access-826dj\") pod \"downloads-5bc67dcd46-c7vr5\" (UID: \"397f1417-0b0a-4d74-94c1-704f3b33eee5\") " pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:05.398021 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.397972 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:05.531993 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.531820 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-c7vr5"] Jul 21 08:24:05.534596 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:24:05.534560 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397f1417_0b0a_4d74_94c1_704f3b33eee5.slice/crio-067ba4a80c54ffe3002928d4b8e3e79a661c1d91d2fc6701a7318bf437e1ab2d WatchSource:0}: Error finding container 067ba4a80c54ffe3002928d4b8e3e79a661c1d91d2fc6701a7318bf437e1ab2d: Status 404 returned error can't find the container with id 067ba4a80c54ffe3002928d4b8e3e79a661c1d91d2fc6701a7318bf437e1ab2d Jul 21 08:24:05.897726 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:05.897676 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-c7vr5" event={"ID":"397f1417-0b0a-4d74-94c1-704f3b33eee5","Type":"ContainerStarted","Data":"067ba4a80c54ffe3002928d4b8e3e79a661c1d91d2fc6701a7318bf437e1ab2d"} Jul 21 08:24:23.974434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:23.974396 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-c7vr5" event={"ID":"397f1417-0b0a-4d74-94c1-704f3b33eee5","Type":"ContainerStarted","Data":"8f038e3352df6744ad4ea0dd2de969d65211d876777bbcf571e7045371fc060c"} Jul 21 08:24:23.974903 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:23.974566 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:23.988256 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:23.988225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-c7vr5" Jul 21 08:24:23.992158 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:24:23.992097 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-c7vr5" podStartSLOduration=1.049411888 podStartE2EDuration="18.992075643s" totalImagesPullingTime="17.942663755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:24:23.990428999 +0000 UTC m=+326.653698234" watchObservedRunningTime="2026-07-21 08:24:23.992075643 +0000 UTC m=+326.655344869" Jul 21 08:25:11.708326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:11.708285 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:25:11.710813 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:11.710786 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71ad86f-e6b6-45b0-87dc-63af669ddfff-metrics-certs\") pod \"network-metrics-daemon-zfdwm\" (UID: \"d71ad86f-e6b6-45b0-87dc-63af669ddfff\") " pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:25:11.831152 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:11.831124 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bh656\"" Jul 21 08:25:11.839091 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:11.839068 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zfdwm" Jul 21 08:25:12.001649 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.001622 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zfdwm"] Jul 21 08:25:12.004340 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:25:12.004315 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71ad86f_e6b6_45b0_87dc_63af669ddfff.slice/crio-02b94b0c8c1a1ec42b19ccc6c2b17e16b34f8ad56245d350155095954d902308 WatchSource:0}: Error finding container 02b94b0c8c1a1ec42b19ccc6c2b17e16b34f8ad56245d350155095954d902308: Status 404 returned error can't find the container with id 02b94b0c8c1a1ec42b19ccc6c2b17e16b34f8ad56245d350155095954d902308 Jul 21 08:25:12.171072 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.171032 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zfdwm" event={"ID":"d71ad86f-e6b6-45b0-87dc-63af669ddfff","Type":"ContainerStarted","Data":"02b94b0c8c1a1ec42b19ccc6c2b17e16b34f8ad56245d350155095954d902308"} Jul 21 08:25:12.507804 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.507774 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:12.508093 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508069 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="alertmanager" containerID="cri-o://adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" gracePeriod=120 Jul 21 08:25:12.508207 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508168 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-web" containerID="cri-o://30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" gracePeriod=120 Jul 21 08:25:12.508207 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508194 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="config-reloader" containerID="cri-o://bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" gracePeriod=120 Jul 21 08:25:12.508398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508158 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-metric" containerID="cri-o://a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" gracePeriod=120 Jul 21 08:25:12.508398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508337 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="prom-label-proxy" containerID="cri-o://4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" gracePeriod=120 Jul 21 08:25:12.508398 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:12.508349 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy" containerID="cri-o://a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" gracePeriod=120 Jul 21 08:25:13.179019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.178926 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" exitCode=0 Jul 21 08:25:13.179019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.178953 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" exitCode=0 Jul 21 08:25:13.179019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.178960 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" exitCode=0 Jul 21 08:25:13.179019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.178965 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" exitCode=0 Jul 21 08:25:13.179019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.178997 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb"} Jul 21 08:25:13.179619 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.179098 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1"} Jul 21 08:25:13.179619 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.179137 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad"} Jul 21 08:25:13.179619 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.179167 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9"} Jul 21 08:25:13.180811 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.180781 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zfdwm" event={"ID":"d71ad86f-e6b6-45b0-87dc-63af669ddfff","Type":"ContainerStarted","Data":"8e2f494614bee0ecf303338ba0cbd08ff0c3f2c5f7137a812d9163258230bd98"} Jul 21 08:25:13.180938 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.180843 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zfdwm" event={"ID":"d71ad86f-e6b6-45b0-87dc-63af669ddfff","Type":"ContainerStarted","Data":"85bb0ae9063969748e0ec3a16537a8da34beea691c37821fd2de077c61ec9c12"} Jul 21 08:25:13.196384 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.196324 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zfdwm" podStartSLOduration=375.305207961 podStartE2EDuration="6m16.196304639s" totalImagesPullingTime="891.096678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:25:13.194759891 +0000 UTC m=+375.858029117" watchObservedRunningTime="2026-07-21 08:25:13.196304639 +0000 UTC m=+375.859573882" Jul 21 08:25:13.760410 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.760375 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:13.825306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825275 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825311 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825361 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825396 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825431 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825706 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825580 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825706 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825635 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825706 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825668 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825858 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825719 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825858 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825717 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 08:25:13.825858 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825752 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825858 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825815 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.825858 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825832 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:25:13.826085 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825869 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.826085 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.825908 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets\") pod \"632a76c5-52f3-42da-bf23-3928fef288b0\" (UID: \"632a76c5-52f3-42da-bf23-3928fef288b0\") " Jul 21 08:25:13.826183 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.826141 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-main-db\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.826183 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.826158 2569 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-metrics-client-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.827396 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.827364 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:25:13.828804 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.828779 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.829928 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.829846 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.829928 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.829713 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg" (OuterVolumeSpecName: "kube-api-access-2btwg") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "kube-api-access-2btwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:25:13.829928 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.829779 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:25:13.830136 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.829933 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.830280 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.830249 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume" (OuterVolumeSpecName: "config-volume") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.831227 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.831185 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out" (OuterVolumeSpecName: "config-out") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 08:25:13.831851 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.831811 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.834845 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.834789 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.841924 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.841850 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config" (OuterVolumeSpecName: "web-config") pod "632a76c5-52f3-42da-bf23-3928fef288b0" (UID: "632a76c5-52f3-42da-bf23-3928fef288b0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:25:13.926709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926643 2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-tls-assets\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.926709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926673 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.926709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926699 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-main-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.926709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926707 2569 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-cluster-tls-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.926709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926716 2569 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-config-volume\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926726 2569 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/632a76c5-52f3-42da-bf23-3928fef288b0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926734 2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-web-config\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926742 2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/632a76c5-52f3-42da-bf23-3928fef288b0-config-out\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926750 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926759 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2btwg\" (UniqueName: \"kubernetes.io/projected/632a76c5-52f3-42da-bf23-3928fef288b0-kube-api-access-2btwg\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:13.927001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:13.926767 2569 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/632a76c5-52f3-42da-bf23-3928fef288b0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:25:14.187673 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187591 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" exitCode=0 Jul 21 08:25:14.187673 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187616 2569 generic.go:356] "Generic (PLEG): container finished" podID="632a76c5-52f3-42da-bf23-3928fef288b0" containerID="30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" exitCode=0 Jul 21 08:25:14.188133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187672 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.188133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187672 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be"} Jul 21 08:25:14.188133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187765 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94"} Jul 21 08:25:14.188133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187801 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"632a76c5-52f3-42da-bf23-3928fef288b0","Type":"ContainerDied","Data":"be80c082cb6e9bbb3cd4a086f35ad46b684c3258e68fa1193919f7614b89dd1d"} Jul 21 08:25:14.188133 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.187831 2569 scope.go:117] "RemoveContainer" containerID="4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" Jul 21 08:25:14.196164 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.196147 2569 scope.go:117] "RemoveContainer" containerID="a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" Jul 21 08:25:14.203597 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.203579 2569 scope.go:117] "RemoveContainer" containerID="a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" Jul 21 08:25:14.209966 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.209941 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:14.211715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.211692 2569 scope.go:117] "RemoveContainer" containerID="30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" Jul 21 08:25:14.214248 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.214228 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:14.219449 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.219429 2569 scope.go:117] "RemoveContainer" containerID="bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" Jul 21 08:25:14.227442 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.227422 2569 scope.go:117] "RemoveContainer" containerID="adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" Jul 21 08:25:14.235823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.235785 2569 scope.go:117] "RemoveContainer" containerID="b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767" Jul 21 08:25:14.237262 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237244 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:14.237523 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237510 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="prom-label-proxy" Jul 21 08:25:14.237560 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237525 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="prom-label-proxy" Jul 21 08:25:14.237560 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237535 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-metric" Jul 21 08:25:14.237560 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237540 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-metric" Jul 21 08:25:14.237560 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237551 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="init-config-reloader" Jul 21 08:25:14.237560 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237556 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="init-config-reloader" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237564 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237569 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237579 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="alertmanager" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237584 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="alertmanager" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237590 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-web" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237604 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-web" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237613 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="config-reloader" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237619 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="config-reloader" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237665 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="config-reloader" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237672 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237703 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="prom-label-proxy" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237714 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-metric" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237721 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="kube-rbac-proxy-web" Jul 21 08:25:14.237782 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.237726 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" containerName="alertmanager" Jul 21 08:25:14.241288 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.241266 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.243716 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.243669 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 21 08:25:14.243833 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.243669 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 21 08:25:14.243895 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.243832 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 21 08:25:14.244009 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.243995 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 21 08:25:14.244078 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244015 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 21 08:25:14.244078 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244036 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zgctx\"" Jul 21 08:25:14.244409 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244395 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 21 08:25:14.244496 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244417 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 21 08:25:14.244496 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244398 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 21 08:25:14.244662 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244641 2569 scope.go:117] "RemoveContainer" containerID="4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" Jul 21 08:25:14.244968 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.244945 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb\": container with ID starting with 4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb not found: ID does not exist" containerID="4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" Jul 21 08:25:14.245061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.244978 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb"} err="failed to get container status \"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb\": rpc error: code = NotFound desc = could not find container \"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb\": container with ID starting with 4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb not found: ID does not exist" Jul 21 08:25:14.245061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.245002 2569 scope.go:117] "RemoveContainer" containerID="a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" Jul 21 08:25:14.245308 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.245287 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be\": container with ID starting with a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be not found: ID does not exist" containerID="a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" Jul 21 08:25:14.245378 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.245313 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be"} err="failed to get container status \"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be\": rpc error: code = NotFound desc = could not find container \"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be\": container with ID starting with a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be not found: ID does not exist" Jul 21 08:25:14.245378 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.245330 2569 scope.go:117] "RemoveContainer" containerID="a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" Jul 21 08:25:14.245650 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.245633 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1\": container with ID starting with a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1 not found: ID does not exist" containerID="a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" Jul 21 08:25:14.245741 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.245656 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1"} err="failed to get container status \"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1\": rpc error: code = NotFound desc = could not find container \"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1\": container with ID starting with a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1 not found: ID does not exist" Jul 21 08:25:14.245788 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.245673 2569 scope.go:117] "RemoveContainer" containerID="30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" Jul 21 08:25:14.246027 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.246012 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94\": container with ID starting with 30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94 not found: ID does not exist" containerID="30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" Jul 21 08:25:14.246067 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246031 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94"} err="failed to get container status \"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94\": rpc error: code = NotFound desc = could not find container \"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94\": container with ID starting with 30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94 not found: ID does not exist" Jul 21 08:25:14.246067 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246050 2569 scope.go:117] "RemoveContainer" containerID="bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" Jul 21 08:25:14.246397 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.246369 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad\": container with ID starting with bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad not found: ID does not exist" containerID="bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" Jul 21 08:25:14.246501 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246391 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad"} err="failed to get container status \"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad\": rpc error: code = NotFound desc = could not find container \"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad\": container with ID starting with bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad not found: ID does not exist" Jul 21 08:25:14.246501 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246410 2569 scope.go:117] "RemoveContainer" containerID="adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" Jul 21 08:25:14.246715 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.246694 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9\": container with ID starting with adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9 not found: ID does not exist" containerID="adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" Jul 21 08:25:14.246805 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246724 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9"} err="failed to get container status \"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9\": rpc error: code = NotFound desc = could not find container \"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9\": container with ID starting with adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9 not found: ID does not exist" Jul 21 08:25:14.246805 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.246744 2569 scope.go:117] "RemoveContainer" containerID="b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767" Jul 21 08:25:14.247056 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:25:14.247035 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767\": container with ID starting with b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767 not found: ID does not exist" containerID="b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767" Jul 21 08:25:14.247135 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247064 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767"} err="failed to get container status \"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767\": rpc error: code = NotFound desc = could not find container \"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767\": container with ID starting with b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767 not found: ID does not exist" Jul 21 08:25:14.247135 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247085 2569 scope.go:117] "RemoveContainer" containerID="4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb" Jul 21 08:25:14.247360 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247340 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb"} err="failed to get container status \"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb\": rpc error: code = NotFound desc = could not find container \"4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb\": container with ID starting with 4c43ced3889d34a94e7d4a74e5611de3773b8a895954c48dec7e121913a053eb not found: ID does not exist" Jul 21 08:25:14.247441 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247360 2569 scope.go:117] "RemoveContainer" containerID="a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be" Jul 21 08:25:14.247612 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247579 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be"} err="failed to get container status \"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be\": rpc error: code = NotFound desc = could not find container \"a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be\": container with ID starting with a333ea285fd2c5aa35e9502c67ef901613a343028c75a0a875d423ff40a365be not found: ID does not exist" Jul 21 08:25:14.247724 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247616 2569 scope.go:117] "RemoveContainer" containerID="a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1" Jul 21 08:25:14.247937 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247914 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1"} err="failed to get container status \"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1\": rpc error: code = NotFound desc = could not find container \"a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1\": container with ID starting with a3d80d3dc2c013a951572a085d0156387f82f6c4b1f930ba4f893bd3ba176ba1 not found: ID does not exist" Jul 21 08:25:14.247991 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.247940 2569 scope.go:117] "RemoveContainer" containerID="30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94" Jul 21 08:25:14.248272 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248248 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94"} err="failed to get container status \"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94\": rpc error: code = NotFound desc = could not find container \"30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94\": container with ID starting with 30d0019343c011beed50bab51ad56cc298b0359785c6f4e5e74bf0ee2a666f94 not found: ID does not exist" Jul 21 08:25:14.248370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248274 2569 scope.go:117] "RemoveContainer" containerID="bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad" Jul 21 08:25:14.248571 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248547 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad"} err="failed to get container status \"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad\": rpc error: code = NotFound desc = could not find container \"bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad\": container with ID starting with bd51e9010cfe4e4fda9fad55c6d0e20a203bb16b627498b52c05ad4b879fd2ad not found: ID does not exist" Jul 21 08:25:14.248659 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248573 2569 scope.go:117] "RemoveContainer" containerID="adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9" Jul 21 08:25:14.248939 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248891 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9"} err="failed to get container status \"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9\": rpc error: code = NotFound desc = could not find container \"adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9\": container with ID starting with adcf1d7cee5a053fb9bd178869e5ba4e99162b10919afc671c77c9067eab9be9 not found: ID does not exist" Jul 21 08:25:14.248939 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.248936 2569 scope.go:117] "RemoveContainer" containerID="b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767" Jul 21 08:25:14.249243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.249226 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 21 08:25:14.249328 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.249234 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767"} err="failed to get container status \"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767\": rpc error: code = NotFound desc = could not find container \"b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767\": container with ID starting with b9145f0450bfc2c83e448b7f5b6707fad05735b931f1053a80566b498e399767 not found: ID does not exist" Jul 21 08:25:14.254012 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.253990 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:14.330711 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330643 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330711 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330704 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330912 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330758 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-config-out\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330912 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330821 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330912 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330844 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-web-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330912 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330863 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.330912 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330878 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330923 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330956 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.330986 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k9kr\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-kube-api-access-6k9kr\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.331012 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331061 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.331035 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.331197 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.331056 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432016 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.431975 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-web-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432016 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432020 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432045 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432079 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432110 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k9kr\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-kube-api-access-6k9kr\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432181 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432218 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432246 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432237 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432255 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432273 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432308 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-config-out\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432334 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.432761 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432712 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.433028 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.432969 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.433136 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.433029 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad07d97-7a48-48e8-9974-493b640cb220-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435498 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435378 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435498 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435442 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435569 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435569 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-config-volume\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435795 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435760 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ad07d97-7a48-48e8-9974-493b640cb220-config-out\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435894 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-web-config\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435932 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.435932 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.435913 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.437440 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.437419 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7ad07d97-7a48-48e8-9974-493b640cb220-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.439717 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.439649 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k9kr\" (UniqueName: \"kubernetes.io/projected/7ad07d97-7a48-48e8-9974-493b640cb220-kube-api-access-6k9kr\") pod \"alertmanager-main-0\" (UID: \"7ad07d97-7a48-48e8-9974-493b640cb220\") " pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.551950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.551913 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 21 08:25:14.704564 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:14.704181 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 21 08:25:14.706624 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:25:14.706593 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad07d97_7a48_48e8_9974_493b640cb220.slice/crio-9cae4a6d39c48356ba0fe7b01f3a37b9b12f45b354cebed2e9b1442f1ca8501f WatchSource:0}: Error finding container 9cae4a6d39c48356ba0fe7b01f3a37b9b12f45b354cebed2e9b1442f1ca8501f: Status 404 returned error can't find the container with id 9cae4a6d39c48356ba0fe7b01f3a37b9b12f45b354cebed2e9b1442f1ca8501f Jul 21 08:25:15.193942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:15.193905 2569 generic.go:356] "Generic (PLEG): container finished" podID="7ad07d97-7a48-48e8-9974-493b640cb220" containerID="bb875964326269fd24002bf557e111632dd1229b1dde3f3d0458288eead71167" exitCode=0 Jul 21 08:25:15.194365 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:15.193980 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerDied","Data":"bb875964326269fd24002bf557e111632dd1229b1dde3f3d0458288eead71167"} Jul 21 08:25:15.194365 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:15.194035 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"9cae4a6d39c48356ba0fe7b01f3a37b9b12f45b354cebed2e9b1442f1ca8501f"} Jul 21 08:25:15.834013 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:15.833977 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="632a76c5-52f3-42da-bf23-3928fef288b0" path="/var/lib/kubelet/pods/632a76c5-52f3-42da-bf23-3928fef288b0/volumes" Jul 21 08:25:16.201307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201209 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"1a06748dc85d23212ae18ee661c1523dcbfcc672a31d10bd989a2aacdcb0137b"} Jul 21 08:25:16.201307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201274 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"99f98d0daa907ef563ec0a093be86810a8613205425752cfb059f5091ec4af68"} Jul 21 08:25:16.201307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201304 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"381587a29b4a10ee94b570a4537281f72f11a5cef9a6818ab4a27aaef97453c1"} Jul 21 08:25:16.201812 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201328 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"646b5a86664cfade901d1e30ddda20d1e28f1891fc7623fed1b750381c748768"} Jul 21 08:25:16.201812 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201351 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"f574cdd47dcb05fc940c16b91d325029b2d639145284d883800b2d076c4460de"} Jul 21 08:25:16.201812 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.201375 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7ad07d97-7a48-48e8-9974-493b640cb220","Type":"ContainerStarted","Data":"e6d687b38966dc903fbe4e33de6f9169c0ed119faaa0efe228ddeff51fe71d76"} Jul 21 08:25:16.225954 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.225894 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.208837599 podStartE2EDuration="2.225874436s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.036837ms" isStatefulPod=false podCreationTimestamp="2026-07-21 08:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:25:16.224476045 +0000 UTC m=+378.887745270" watchObservedRunningTime="2026-07-21 08:25:16.225874436 +0000 UTC m=+378.889143664" Jul 21 08:25:16.501324 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.501276 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78cb5854f7-88c5z"] Jul 21 08:25:16.503948 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.503922 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.506325 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506292 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 21 08:25:16.506325 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506312 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 21 08:25:16.506325 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506320 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xdwd9\"" Jul 21 08:25:16.506593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506329 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 21 08:25:16.506593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506367 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 21 08:25:16.506593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.506293 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 21 08:25:16.510796 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.510773 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 21 08:25:16.519913 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.519879 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78cb5854f7-88c5z"] Jul 21 08:25:16.550859 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.550823 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-federate-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.550859 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.550864 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.550890 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pjfk\" (UniqueName: \"kubernetes.io/projected/956e0144-e6cd-4acd-8d40-d95fd63fcdae-kube-api-access-2pjfk\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.550973 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.551002 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.551066 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-metrics-client-ca\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.551082 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.551262 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.551105 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-serving-certs-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652325 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-metrics-client-ca\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652373 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652404 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-serving-certs-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652440 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-federate-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652467 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652523 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pjfk\" (UniqueName: \"kubernetes.io/projected/956e0144-e6cd-4acd-8d40-d95fd63fcdae-kube-api-access-2pjfk\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652576 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.652642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.652621 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.653316 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.653288 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-serving-certs-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.653572 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.653462 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.653843 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.653819 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/956e0144-e6cd-4acd-8d40-d95fd63fcdae-metrics-client-ca\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.655346 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.655323 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.655531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.655513 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-federate-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.655607 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.655589 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-telemeter-client-tls\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.655758 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.655738 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/956e0144-e6cd-4acd-8d40-d95fd63fcdae-secret-telemeter-client\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.664580 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.662111 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pjfk\" (UniqueName: \"kubernetes.io/projected/956e0144-e6cd-4acd-8d40-d95fd63fcdae-kube-api-access-2pjfk\") pod \"telemeter-client-78cb5854f7-88c5z\" (UID: \"956e0144-e6cd-4acd-8d40-d95fd63fcdae\") " pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.815923 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.815825 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" Jul 21 08:25:16.966390 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:16.966357 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78cb5854f7-88c5z"] Jul 21 08:25:16.969774 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:25:16.969748 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956e0144_e6cd_4acd_8d40_d95fd63fcdae.slice/crio-ce3e4e1e83073511b49104932c09a3d6b6182d94443aa7900f9fa722a7825434 WatchSource:0}: Error finding container ce3e4e1e83073511b49104932c09a3d6b6182d94443aa7900f9fa722a7825434: Status 404 returned error can't find the container with id ce3e4e1e83073511b49104932c09a3d6b6182d94443aa7900f9fa722a7825434 Jul 21 08:25:17.207520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:17.207430 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" event={"ID":"956e0144-e6cd-4acd-8d40-d95fd63fcdae","Type":"ContainerStarted","Data":"ce3e4e1e83073511b49104932c09a3d6b6182d94443aa7900f9fa722a7825434"} Jul 21 08:25:19.218827 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:19.218784 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" event={"ID":"956e0144-e6cd-4acd-8d40-d95fd63fcdae","Type":"ContainerStarted","Data":"62b90a0ca796424cab8462f22a4575cfad368b7dc94e738026482306fe55b78a"} Jul 21 08:25:19.219257 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:19.218863 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" event={"ID":"956e0144-e6cd-4acd-8d40-d95fd63fcdae","Type":"ContainerStarted","Data":"9b568155b980576e97feef28ea199a3e452a8adf6bd447ee8f96d9c500536238"} Jul 21 08:25:19.219257 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:19.218891 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" event={"ID":"956e0144-e6cd-4acd-8d40-d95fd63fcdae","Type":"ContainerStarted","Data":"d4239556878698aca4f29772fca7ef0051685f55694678f397cbdaab79e54b09"} Jul 21 08:25:19.239332 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:25:19.239261 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78cb5854f7-88c5z" podStartSLOduration=1.553617807 podStartE2EDuration="3.239238582s" totalImagesPullingTime="1.685620775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:25:19.237414115 +0000 UTC m=+381.900683340" watchObservedRunningTime="2026-07-21 08:25:19.239238582 +0000 UTC m=+381.902507815" Jul 21 08:28:26.126984 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.126948 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-clzfg"] Jul 21 08:28:26.130122 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.130099 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.132767 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.132741 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 08:28:26.132907 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.132785 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 08:28:26.132907 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.132799 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nv27w\"" Jul 21 08:28:26.140625 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.140600 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-clzfg"] Jul 21 08:28:26.239934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.239897 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2776q\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-kube-api-access-2776q\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.239934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.239938 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-bound-sa-token\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.341394 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.341359 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2776q\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-kube-api-access-2776q\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.341394 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.341399 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-bound-sa-token\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.350869 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.350826 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-bound-sa-token\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.351015 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.350979 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2776q\" (UniqueName: \"kubernetes.io/projected/d7d3b469-d18c-49b1-ba51-08615f1ceb09-kube-api-access-2776q\") pod \"cert-manager-55b4d954d5-clzfg\" (UID: \"d7d3b469-d18c-49b1-ba51-08615f1ceb09\") " pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.439995 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.439894 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-clzfg" Jul 21 08:28:26.578803 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.578780 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-clzfg"] Jul 21 08:28:26.581441 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:28:26.581395 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7d3b469_d18c_49b1_ba51_08615f1ceb09.slice/crio-c80cdaf0066a9136b4bc380630bbf3b7270db462492632cf63df5fc2d931c167 WatchSource:0}: Error finding container c80cdaf0066a9136b4bc380630bbf3b7270db462492632cf63df5fc2d931c167: Status 404 returned error can't find the container with id c80cdaf0066a9136b4bc380630bbf3b7270db462492632cf63df5fc2d931c167 Jul 21 08:28:26.964746 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:26.964707 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-clzfg" event={"ID":"d7d3b469-d18c-49b1-ba51-08615f1ceb09","Type":"ContainerStarted","Data":"c80cdaf0066a9136b4bc380630bbf3b7270db462492632cf63df5fc2d931c167"} Jul 21 08:28:29.979595 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:29.979497 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-clzfg" event={"ID":"d7d3b469-d18c-49b1-ba51-08615f1ceb09","Type":"ContainerStarted","Data":"ee719520db46eb7c7d71e5b26fb1fc967ea9d77d55238c4e709454c4a73c420f"} Jul 21 08:28:29.995534 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:29.995484 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-clzfg" podStartSLOduration=0.99702617 podStartE2EDuration="3.995465378s" totalImagesPullingTime="2.998439208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:28:29.994469175 +0000 UTC m=+572.657738400" watchObservedRunningTime="2026-07-21 08:28:29.995465378 +0000 UTC m=+572.658734603" Jul 21 08:28:31.348343 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.346073 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt"] Jul 21 08:28:31.350506 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.350478 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.353140 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.353111 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 08:28:31.353916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.353897 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 08:28:31.354175 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.354153 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nb55l\"" Jul 21 08:28:31.355154 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.355129 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt"] Jul 21 08:28:31.486303 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.486260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.486497 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.486369 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqtkc\" (UniqueName: \"kubernetes.io/projected/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-kube-api-access-nqtkc\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.586983 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.586940 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqtkc\" (UniqueName: \"kubernetes.io/projected/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-kube-api-access-nqtkc\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.587189 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.587002 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.587403 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.587380 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.595536 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.595508 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqtkc\" (UniqueName: \"kubernetes.io/projected/fc85a4c3-c92f-4da6-99eb-1c6dc83abf19-kube-api-access-nqtkc\") pod \"openshift-lws-operator-fd8ccff4c-9wswt\" (UID: \"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.661591 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.661490 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" Jul 21 08:28:31.800853 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.800821 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt"] Jul 21 08:28:31.804165 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:28:31.804134 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc85a4c3_c92f_4da6_99eb_1c6dc83abf19.slice/crio-37899374575b9ebe4a6883bc201182cd754e00c81b364b60578aec432d6dd6a9 WatchSource:0}: Error finding container 37899374575b9ebe4a6883bc201182cd754e00c81b364b60578aec432d6dd6a9: Status 404 returned error can't find the container with id 37899374575b9ebe4a6883bc201182cd754e00c81b364b60578aec432d6dd6a9 Jul 21 08:28:31.989083 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:31.988996 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" event={"ID":"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19","Type":"ContainerStarted","Data":"37899374575b9ebe4a6883bc201182cd754e00c81b364b60578aec432d6dd6a9"} Jul 21 08:28:35.004035 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:35.003998 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" event={"ID":"fc85a4c3-c92f-4da6-99eb-1c6dc83abf19","Type":"ContainerStarted","Data":"76d020eddddd6e71354dc62933b95d250db6a11e066f6db917273e9e32ef75a5"} Jul 21 08:28:35.019936 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:35.019885 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9wswt" podStartSLOduration=1.128450327 podStartE2EDuration="4.019865576s" totalImagesPullingTime="2.891415249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:28:35.019488238 +0000 UTC m=+577.682757476" watchObservedRunningTime="2026-07-21 08:28:35.019865576 +0000 UTC m=+577.683134803" Jul 21 08:28:50.184356 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.184312 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7686776559-hnscw"] Jul 21 08:28:50.187649 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.187625 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.191622 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.191592 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 08:28:50.191819 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.191622 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fmjmb\"" Jul 21 08:28:50.191819 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.191623 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 08:28:50.191819 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.191623 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 08:28:50.195719 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.195675 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7686776559-hnscw"] Jul 21 08:28:50.351692 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.351654 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa191af6-81b7-499f-b68a-df04da73022f-manager-config\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.351904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.351724 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwrqb\" (UniqueName: \"kubernetes.io/projected/aa191af6-81b7-499f-b68a-df04da73022f-kube-api-access-lwrqb\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.351904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.351789 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-metrics-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.351904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.351858 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.453174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.453079 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwrqb\" (UniqueName: \"kubernetes.io/projected/aa191af6-81b7-499f-b68a-df04da73022f-kube-api-access-lwrqb\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.453174 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.453126 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-metrics-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.453397 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.453178 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.453397 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.453261 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa191af6-81b7-499f-b68a-df04da73022f-manager-config\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.453937 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.453912 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aa191af6-81b7-499f-b68a-df04da73022f-manager-config\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.456227 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.456202 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-metrics-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.456342 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.456205 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa191af6-81b7-499f-b68a-df04da73022f-cert\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.464636 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.464608 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwrqb\" (UniqueName: \"kubernetes.io/projected/aa191af6-81b7-499f-b68a-df04da73022f-kube-api-access-lwrqb\") pod \"lws-controller-manager-7686776559-hnscw\" (UID: \"aa191af6-81b7-499f-b68a-df04da73022f\") " pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.497495 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.497454 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:50.640831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:50.640744 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7686776559-hnscw"] Jul 21 08:28:50.643188 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:28:50.643160 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa191af6_81b7_499f_b68a_df04da73022f.slice/crio-28a32332a0a8173e3a12bce707369f727a2f7ad6d654445d8d5d46ba595eba07 WatchSource:0}: Error finding container 28a32332a0a8173e3a12bce707369f727a2f7ad6d654445d8d5d46ba595eba07: Status 404 returned error can't find the container with id 28a32332a0a8173e3a12bce707369f727a2f7ad6d654445d8d5d46ba595eba07 Jul 21 08:28:51.071312 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:51.071277 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" event={"ID":"aa191af6-81b7-499f-b68a-df04da73022f","Type":"ContainerStarted","Data":"28a32332a0a8173e3a12bce707369f727a2f7ad6d654445d8d5d46ba595eba07"} Jul 21 08:28:53.080956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:53.080922 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" event={"ID":"aa191af6-81b7-499f-b68a-df04da73022f","Type":"ContainerStarted","Data":"6ea99b001b827d25d5173901e18bbf3b39e53c3acbc55db7faea707fff4fc63b"} Jul 21 08:28:53.081353 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:53.080986 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:28:53.101474 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:53.101419 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" podStartSLOduration=1.331662182 podStartE2EDuration="3.101400731s" totalImagesPullingTime="1.769738549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:28:53.100336081 +0000 UTC m=+595.763605311" watchObservedRunningTime="2026-07-21 08:28:53.101400731 +0000 UTC m=+595.764669956" Jul 21 08:28:54.761784 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.761749 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66"] Jul 21 08:28:54.765241 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.765220 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.769740 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.769713 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-568nx\"" Jul 21 08:28:54.769900 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.769838 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 08:28:54.770001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.769983 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 08:28:54.770001 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.769994 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 08:28:54.770320 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.770304 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 08:28:54.787641 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.787611 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66"] Jul 21 08:28:54.792266 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.792229 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.792431 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.792278 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-manifests\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.792431 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.792299 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z25hv\" (UniqueName: \"kubernetes.io/projected/20479057-0345-42d2-934d-96620bb4ffe1-kube-api-access-z25hv\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.792431 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.792372 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-webhook-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.792431 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.792429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-tmp\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.893731 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.893655 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.893924 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.893774 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-manifests\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.893924 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.893800 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z25hv\" (UniqueName: \"kubernetes.io/projected/20479057-0345-42d2-934d-96620bb4ffe1-kube-api-access-z25hv\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.893924 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.893845 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-webhook-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.893924 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.893878 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-tmp\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.894399 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.894369 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-manifests\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.896717 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.896671 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-webhook-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.896831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.896805 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/20479057-0345-42d2-934d-96620bb4ffe1-tmp\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.896931 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.896915 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/20479057-0345-42d2-934d-96620bb4ffe1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:54.906847 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:54.906812 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z25hv\" (UniqueName: \"kubernetes.io/projected/20479057-0345-42d2-934d-96620bb4ffe1-kube-api-access-z25hv\") pod \"opendatahub-operator-controller-manager-787895fdf5-s8g66\" (UID: \"20479057-0345-42d2-934d-96620bb4ffe1\") " pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:55.076646 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:55.076561 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:28:55.218712 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:55.218670 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66"] Jul 21 08:28:55.221160 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:28:55.221130 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20479057_0345_42d2_934d_96620bb4ffe1.slice/crio-97b16cb1300b8c7acf207412f7ac0eb1c6dd3f64122742d22a567cb40f4a6744 WatchSource:0}: Error finding container 97b16cb1300b8c7acf207412f7ac0eb1c6dd3f64122742d22a567cb40f4a6744: Status 404 returned error can't find the container with id 97b16cb1300b8c7acf207412f7ac0eb1c6dd3f64122742d22a567cb40f4a6744 Jul 21 08:28:56.097658 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:56.097607 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" event={"ID":"20479057-0345-42d2-934d-96620bb4ffe1","Type":"ContainerStarted","Data":"97b16cb1300b8c7acf207412f7ac0eb1c6dd3f64122742d22a567cb40f4a6744"} Jul 21 08:28:58.455820 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:58.455791 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:28:58.455820 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:58.455821 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:28:58.464566 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:58.464539 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:28:58.464566 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:58.464567 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:28:59.114549 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:28:59.114509 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" event={"ID":"20479057-0345-42d2-934d-96620bb4ffe1","Type":"ContainerStarted","Data":"eff7c70a7792f855791dbed069e07065c7b17512f8cc52dd980a8b7a733c6570"} Jul 21 08:29:00.121380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:00.121342 2569 generic.go:356] "Generic (PLEG): container finished" podID="20479057-0345-42d2-934d-96620bb4ffe1" containerID="eff7c70a7792f855791dbed069e07065c7b17512f8cc52dd980a8b7a733c6570" exitCode=0 Jul 21 08:29:00.121810 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:00.121416 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" event={"ID":"20479057-0345-42d2-934d-96620bb4ffe1","Type":"ContainerDied","Data":"eff7c70a7792f855791dbed069e07065c7b17512f8cc52dd980a8b7a733c6570"} Jul 21 08:29:00.121983 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:00.121968 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:29:01.129124 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:01.129087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" event={"ID":"20479057-0345-42d2-934d-96620bb4ffe1","Type":"ContainerStarted","Data":"59264fe509ea95377abd16233f752969a2a9804469e87314f57a76a2e6ed6e6c"} Jul 21 08:29:01.129622 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:01.129225 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:29:01.148917 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:01.148867 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" podStartSLOduration=2.561354432 podStartE2EDuration="7.148848684s" totalImagesPullingTime="3.573165895s" totalInitContainerRuntime="1.014328357s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:28:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:29:01.147758602 +0000 UTC m=+603.811027823" watchObservedRunningTime="2026-07-21 08:29:01.148848684 +0000 UTC m=+603.812117910" Jul 21 08:29:04.087943 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:04.087911 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7686776559-hnscw" Jul 21 08:29:12.136495 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:12.136456 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-787895fdf5-s8g66" Jul 21 08:29:21.827168 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.827121 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw"] Jul 21 08:29:21.829822 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.829793 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.832915 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.832889 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q8qzp\"" Jul 21 08:29:21.833070 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.832941 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 08:29:21.833207 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.833190 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 08:29:21.833291 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.833208 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 08:29:21.833291 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.833236 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 08:29:21.833724 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.833711 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 08:29:21.833915 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.833902 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 08:29:21.844382 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.844353 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw"] Jul 21 08:29:21.937700 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937636 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.937886 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937711 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/188c5f34-1541-47a6-84ae-bacc24668576-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.937886 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937802 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.937886 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937846 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj55z\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-kube-api-access-xj55z\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.937886 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937870 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.938019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937908 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:21.938019 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:21.937939 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039150 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039103 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039162 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj55z\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-kube-api-access-xj55z\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039183 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039204 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039230 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039264 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.039344 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.039281 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/188c5f34-1541-47a6-84ae-bacc24668576-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.040242 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.040201 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.041896 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.041871 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/188c5f34-1541-47a6-84ae-bacc24668576-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.042036 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.041921 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.042101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.042043 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.042139 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.042118 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/188c5f34-1541-47a6-84ae-bacc24668576-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.050970 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.050939 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj55z\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-kube-api-access-xj55z\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.056777 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.056750 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/188c5f34-1541-47a6-84ae-bacc24668576-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-bb4pw\" (UID: \"188c5f34-1541-47a6-84ae-bacc24668576\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.139898 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.139814 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:22.288641 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:22.288614 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw"] Jul 21 08:29:22.291315 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:29:22.291287 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188c5f34_1541_47a6_84ae_bacc24668576.slice/crio-8c55a8d79dab34752516deb458f50b43ba6982dd34727c2eaf39001a9c362a5b WatchSource:0}: Error finding container 8c55a8d79dab34752516deb458f50b43ba6982dd34727c2eaf39001a9c362a5b: Status 404 returned error can't find the container with id 8c55a8d79dab34752516deb458f50b43ba6982dd34727c2eaf39001a9c362a5b Jul 21 08:29:23.232768 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:23.232725 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" event={"ID":"188c5f34-1541-47a6-84ae-bacc24668576","Type":"ContainerStarted","Data":"8c55a8d79dab34752516deb458f50b43ba6982dd34727c2eaf39001a9c362a5b"} Jul 21 08:29:24.695754 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:24.695631 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:29:24.696156 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:24.695757 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:29:25.244893 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:25.244855 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" event={"ID":"188c5f34-1541-47a6-84ae-bacc24668576","Type":"ContainerStarted","Data":"20fbe37453fb80359e14edad83632698e70547599ccbfc63af5dbbcdf7a4f54f"} Jul 21 08:29:25.245116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:25.245068 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:29:25.246965 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:25.246937 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-bb4pw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 08:29:25.247109 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:25.246994 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" podUID="188c5f34-1541-47a6-84ae-bacc24668576" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 08:29:25.266877 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:25.266800 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" podStartSLOduration=1.864887475 podStartE2EDuration="4.266783544s" totalImagesPullingTime="2.401896069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:29:25.265509545 +0000 UTC m=+627.928778770" watchObservedRunningTime="2026-07-21 08:29:25.266783544 +0000 UTC m=+627.930052769" Jul 21 08:29:26.250690 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:29:26.250659 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-bb4pw" Jul 21 08:31:09.395643 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.395548 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:31:09.399726 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.399698 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.402455 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.402425 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-c7k9p\"" Jul 21 08:31:09.402621 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.402459 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 08:31:09.402621 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.402427 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 08:31:09.413889 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.413863 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:31:09.505831 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.505787 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvcm\" (UniqueName: \"kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.506038 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.505842 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.606413 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.606375 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvcm\" (UniqueName: \"kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.606594 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.606438 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.606896 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.606876 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.615756 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.615730 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvcm\" (UniqueName: \"kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm\") pod \"kuadrant-operator-controller-manager-86878c769f-vqj8w\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.713643 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.713534 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:09.888091 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:09.888065 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:31:09.891181 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:31:09.891143 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5b1157_0366_42b2_b616_8593f377b4bf.slice/crio-1abc37bdea9f1a4009eda5b7faedab6bdd6c16df2b4ad75420c7af65aeffab30 WatchSource:0}: Error finding container 1abc37bdea9f1a4009eda5b7faedab6bdd6c16df2b4ad75420c7af65aeffab30: Status 404 returned error can't find the container with id 1abc37bdea9f1a4009eda5b7faedab6bdd6c16df2b4ad75420c7af65aeffab30 Jul 21 08:31:10.751671 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:10.751604 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" event={"ID":"eb5b1157-0366-42b2-b616-8593f377b4bf","Type":"ContainerStarted","Data":"1abc37bdea9f1a4009eda5b7faedab6bdd6c16df2b4ad75420c7af65aeffab30"} Jul 21 08:31:15.782892 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:15.782853 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" event={"ID":"eb5b1157-0366-42b2-b616-8593f377b4bf","Type":"ContainerStarted","Data":"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2"} Jul 21 08:31:15.783415 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:15.782998 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:31:15.802652 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:15.802594 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" podStartSLOduration=1.9824510850000001 podStartE2EDuration="6.802575099s" totalImagesPullingTime="4.820124014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:31:15.800227886 +0000 UTC m=+738.463497112" watchObservedRunningTime="2026-07-21 08:31:15.802575099 +0000 UTC m=+738.465844324" Jul 21 08:31:26.789990 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:31:26.789953 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:33:32.446098 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.446062 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb"] Jul 21 08:33:32.449643 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.449619 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.452460 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.452434 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 08:33:32.453633 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.453615 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 08:33:32.453772 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.453628 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k878q\"" Jul 21 08:33:32.453772 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.453630 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 21 08:33:32.458898 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.458872 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb"] Jul 21 08:33:32.520310 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520271 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.520476 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520326 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6eedde-c4e6-4268-a753-1c796862c919-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.520476 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520348 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng5m2\" (UniqueName: \"kubernetes.io/projected/0b6eedde-c4e6-4268-a753-1c796862c919-kube-api-access-ng5m2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.520476 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520426 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.520585 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520514 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.520621 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.520583 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621062 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621016 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621084 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621124 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6eedde-c4e6-4268-a753-1c796862c919-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621148 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng5m2\" (UniqueName: \"kubernetes.io/projected/0b6eedde-c4e6-4268-a753-1c796862c919-kube-api-access-ng5m2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621189 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621525 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621459 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621583 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621572 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.621654 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.621625 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.623648 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.623626 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b6eedde-c4e6-4268-a753-1c796862c919-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.623958 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.623938 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b6eedde-c4e6-4268-a753-1c796862c919-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.630218 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.630193 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng5m2\" (UniqueName: \"kubernetes.io/projected/0b6eedde-c4e6-4268-a753-1c796862c919-kube-api-access-ng5m2\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hc6wb\" (UID: \"0b6eedde-c4e6-4268-a753-1c796862c919\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.761552 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.761516 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:32.904478 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:32.904450 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb"] Jul 21 08:33:32.906632 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:33:32.906602 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6eedde_c4e6_4268_a753_1c796862c919.slice/crio-e6d2658c817e50e40104fb6930b0cc09db59ba906713eca2f880ccb558e9f3e8 WatchSource:0}: Error finding container e6d2658c817e50e40104fb6930b0cc09db59ba906713eca2f880ccb558e9f3e8: Status 404 returned error can't find the container with id e6d2658c817e50e40104fb6930b0cc09db59ba906713eca2f880ccb558e9f3e8 Jul 21 08:33:33.439189 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:33.439148 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" event={"ID":"0b6eedde-c4e6-4268-a753-1c796862c919","Type":"ContainerStarted","Data":"e6d2658c817e50e40104fb6930b0cc09db59ba906713eca2f880ccb558e9f3e8"} Jul 21 08:33:34.445215 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:34.445124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" event={"ID":"0b6eedde-c4e6-4268-a753-1c796862c919","Type":"ContainerStarted","Data":"2eea0de52b5716352de474ca4fc4c595fb72ade91d54bbd256c7750a1baa05d2"} Jul 21 08:33:34.461943 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:34.461882 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" podStartSLOduration=1.3415172499999999 podStartE2EDuration="2.461862295s" totalImagesPullingTime="1.120345045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:33:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:33:34.461188495 +0000 UTC m=+877.124457721" watchObservedRunningTime="2026-07-21 08:33:34.461862295 +0000 UTC m=+877.125131520" Jul 21 08:33:42.761989 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:42.761930 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:42.762508 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:42.762021 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:42.774796 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:42.774757 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:43.500985 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:43.500952 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hc6wb" Jul 21 08:33:45.344497 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.344460 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj"] Jul 21 08:33:45.348385 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.348361 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.350956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.350923 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 08:33:45.358148 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.358118 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj"] Jul 21 08:33:45.442372 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.442527 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442383 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.442527 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442481 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.442527 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442523 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.442651 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442544 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjrcs\" (UniqueName: \"kubernetes.io/projected/ee4c7b15-c6b0-4425-9d86-8fb206742962-kube-api-access-vjrcs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.442651 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.442596 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee4c7b15-c6b0-4425-9d86-8fb206742962-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543646 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543604 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543646 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543650 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543717 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543753 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543770 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjrcs\" (UniqueName: \"kubernetes.io/projected/ee4c7b15-c6b0-4425-9d86-8fb206742962-kube-api-access-vjrcs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.543904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.543796 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee4c7b15-c6b0-4425-9d86-8fb206742962-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.544107 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.544080 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.544168 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.544104 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.544241 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.544214 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.546330 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.546310 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee4c7b15-c6b0-4425-9d86-8fb206742962-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.546528 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.546512 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee4c7b15-c6b0-4425-9d86-8fb206742962-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.554268 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.554245 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjrcs\" (UniqueName: \"kubernetes.io/projected/ee4c7b15-c6b0-4425-9d86-8fb206742962-kube-api-access-vjrcs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6zqfj\" (UID: \"ee4c7b15-c6b0-4425-9d86-8fb206742962\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.681430 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.681333 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:45.821461 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:45.821429 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj"] Jul 21 08:33:45.824302 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:33:45.824274 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee4c7b15_c6b0_4425_9d86_8fb206742962.slice/crio-86bb4975dc00a283226c6badb85005a2fddab1510beddb9a06efdb7402edded1 WatchSource:0}: Error finding container 86bb4975dc00a283226c6badb85005a2fddab1510beddb9a06efdb7402edded1: Status 404 returned error can't find the container with id 86bb4975dc00a283226c6badb85005a2fddab1510beddb9a06efdb7402edded1 Jul 21 08:33:46.506462 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:46.506312 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" event={"ID":"ee4c7b15-c6b0-4425-9d86-8fb206742962","Type":"ContainerStarted","Data":"773314ebf7f523f071fc0c5f0cc3df38b6a137dc06250104e8d37068ffea42ca"} Jul 21 08:33:46.506462 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:46.506363 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" event={"ID":"ee4c7b15-c6b0-4425-9d86-8fb206742962","Type":"ContainerStarted","Data":"86bb4975dc00a283226c6badb85005a2fddab1510beddb9a06efdb7402edded1"} Jul 21 08:33:46.524412 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:46.524353 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" podStartSLOduration=1.099597354 podStartE2EDuration="1.524336707s" totalImagesPullingTime="424.739353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:33:46.523022135 +0000 UTC m=+889.186291356" watchObservedRunningTime="2026-07-21 08:33:46.524336707 +0000 UTC m=+889.187605931" Jul 21 08:33:55.682419 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:55.682371 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:55.683098 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:55.682458 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:55.695493 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:55.695464 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:56.565338 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:56.565306 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6zqfj" Jul 21 08:33:58.488136 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:58.488016 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:33:58.492318 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:58.489569 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:33:58.495650 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:58.495623 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:33:58.497257 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:33:58.497237 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:38:58.515081 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:38:58.515050 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:38:58.517640 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:38:58.517625 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:38:58.523165 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:38:58.523146 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:38:58.525504 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:38:58.525487 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:42:05.766559 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:05.766517 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:42:05.767179 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:05.766783 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" podUID="eb5b1157-0366-42b2-b616-8593f377b4bf" containerName="manager" containerID="cri-o://c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2" gracePeriod=10 Jul 21 08:42:08.421752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.421727 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:42:08.557003 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.556920 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume\") pod \"eb5b1157-0366-42b2-b616-8593f377b4bf\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " Jul 21 08:42:08.557003 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.556970 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qvcm\" (UniqueName: \"kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm\") pod \"eb5b1157-0366-42b2-b616-8593f377b4bf\" (UID: \"eb5b1157-0366-42b2-b616-8593f377b4bf\") " Jul 21 08:42:08.557281 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.557256 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eb5b1157-0366-42b2-b616-8593f377b4bf" (UID: "eb5b1157-0366-42b2-b616-8593f377b4bf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 08:42:08.559226 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.559204 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm" (OuterVolumeSpecName: "kube-api-access-9qvcm") pod "eb5b1157-0366-42b2-b616-8593f377b4bf" (UID: "eb5b1157-0366-42b2-b616-8593f377b4bf"). InnerVolumeSpecName "kube-api-access-9qvcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:42:08.657797 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.657759 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qvcm\" (UniqueName: \"kubernetes.io/projected/eb5b1157-0366-42b2-b616-8593f377b4bf-kube-api-access-9qvcm\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:42:08.657797 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.657790 2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb5b1157-0366-42b2-b616-8593f377b4bf-extensions-socket-volume\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:42:08.902936 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.902845 2569 generic.go:356] "Generic (PLEG): container finished" podID="eb5b1157-0366-42b2-b616-8593f377b4bf" containerID="c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2" exitCode=0 Jul 21 08:42:08.902936 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.902897 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" Jul 21 08:42:08.903110 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.902932 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" event={"ID":"eb5b1157-0366-42b2-b616-8593f377b4bf","Type":"ContainerDied","Data":"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2"} Jul 21 08:42:08.903110 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.903013 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w" event={"ID":"eb5b1157-0366-42b2-b616-8593f377b4bf","Type":"ContainerDied","Data":"1abc37bdea9f1a4009eda5b7faedab6bdd6c16df2b4ad75420c7af65aeffab30"} Jul 21 08:42:08.903110 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.903043 2569 scope.go:117] "RemoveContainer" containerID="c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2" Jul 21 08:42:08.913483 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.913460 2569 scope.go:117] "RemoveContainer" containerID="c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2" Jul 21 08:42:08.913815 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:42:08.913796 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2\": container with ID starting with c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2 not found: ID does not exist" containerID="c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2" Jul 21 08:42:08.913879 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.913828 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2"} err="failed to get container status \"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2\": rpc error: code = NotFound desc = could not find container \"c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2\": container with ID starting with c5feb41eea8120bda1e72a1cb7c58a2f3da6216d8498ea06ae5d8f0fb00e6ba2 not found: ID does not exist" Jul 21 08:42:08.941318 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.941285 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:42:08.946258 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:08.946227 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vqj8w"] Jul 21 08:42:09.834063 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:42:09.834026 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb5b1157-0366-42b2-b616-8593f377b4bf" path="/var/lib/kubelet/pods/eb5b1157-0366-42b2-b616-8593f377b4bf/volumes" Jul 21 08:43:07.079466 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.079430 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w"] Jul 21 08:43:07.079893 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.079805 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb5b1157-0366-42b2-b616-8593f377b4bf" containerName="manager" Jul 21 08:43:07.079893 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.079816 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb5b1157-0366-42b2-b616-8593f377b4bf" containerName="manager" Jul 21 08:43:07.079893 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.079878 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb5b1157-0366-42b2-b616-8593f377b4bf" containerName="manager" Jul 21 08:43:07.083631 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.083615 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.087249 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.087224 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 08:43:07.087444 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.087427 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-c7k9p\"" Jul 21 08:43:07.087792 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.087776 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 08:43:07.109161 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.109128 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w"] Jul 21 08:43:07.188785 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.188746 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crlt7\" (UniqueName: \"kubernetes.io/projected/7c93798f-65a1-42e4-be56-d1d776a176ef-kube-api-access-crlt7\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.188970 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.188808 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c93798f-65a1-42e4-be56-d1d776a176ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.289956 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.289919 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c93798f-65a1-42e4-be56-d1d776a176ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.290160 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.290027 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crlt7\" (UniqueName: \"kubernetes.io/projected/7c93798f-65a1-42e4-be56-d1d776a176ef-kube-api-access-crlt7\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.290364 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.290340 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c93798f-65a1-42e4-be56-d1d776a176ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.300130 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.300095 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crlt7\" (UniqueName: \"kubernetes.io/projected/7c93798f-65a1-42e4-be56-d1d776a176ef-kube-api-access-crlt7\") pod \"kuadrant-operator-controller-manager-86878c769f-jxb5w\" (UID: \"7c93798f-65a1-42e4-be56-d1d776a176ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.393895 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.393798 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:07.556537 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:07.556508 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w"] Jul 21 08:43:07.557009 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:43:07.556984 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c93798f_65a1_42e4_be56_d1d776a176ef.slice/crio-5e9b6221d4a2c71eb826468c9491ed203835b9c53eb5e526c7255561494ac887 WatchSource:0}: Error finding container 5e9b6221d4a2c71eb826468c9491ed203835b9c53eb5e526c7255561494ac887: Status 404 returned error can't find the container with id 5e9b6221d4a2c71eb826468c9491ed203835b9c53eb5e526c7255561494ac887 Jul 21 08:43:08.175213 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:08.175171 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" event={"ID":"7c93798f-65a1-42e4-be56-d1d776a176ef","Type":"ContainerStarted","Data":"f6d00aa6f39e32833d1b89465a5be091e8951e03c1cfe7bc2e1186bb58e1b0bf"} Jul 21 08:43:08.175654 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:08.175243 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:08.175654 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:08.175268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" event={"ID":"7c93798f-65a1-42e4-be56-d1d776a176ef","Type":"ContainerStarted","Data":"5e9b6221d4a2c71eb826468c9491ed203835b9c53eb5e526c7255561494ac887"} Jul 21 08:43:08.201350 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:08.201297 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" podStartSLOduration=1.20128057 podStartE2EDuration="1.20128057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:43:08.198789062 +0000 UTC m=+1450.862058287" watchObservedRunningTime="2026-07-21 08:43:08.20128057 +0000 UTC m=+1450.864549793" Jul 21 08:43:19.182637 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:19.182603 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jxb5w" Jul 21 08:43:58.543837 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:58.543720 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:43:58.547300 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:58.547281 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:43:58.552399 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:58.552378 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:43:58.555312 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:43:58.555295 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:45:00.152070 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.152040 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76"] Jul 21 08:45:00.155382 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.155366 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" Jul 21 08:45:00.158547 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.158530 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:45:00.164843 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.164827 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" Jul 21 08:45:00.181603 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.181270 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76"] Jul 21 08:45:00.508484 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.508425 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76"] Jul 21 08:45:00.510809 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:45:00.510782 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd30d57f_75b4_472b_a1c4_3e9c649e45a9.slice/crio-adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a WatchSource:0}: Error finding container adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a: Status 404 returned error can't find the container with id adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a Jul 21 08:45:00.512650 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.512633 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:45:00.698665 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:00.698629 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" event={"ID":"fd30d57f-75b4-472b-a1c4-3e9c649e45a9","Type":"ContainerStarted","Data":"adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a"} Jul 21 08:45:03.716115 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:03.716022 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" event={"ID":"fd30d57f-75b4-472b-a1c4-3e9c649e45a9","Type":"ContainerStarted","Data":"d73cf5e1a31b93d150467781cb31798a38489f15b59498ef62200cf494412f26"} Jul 21 08:45:03.734563 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:03.734506 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" podStartSLOduration=1.1939562989999999 podStartE2EDuration="3.734489311s" totalImagesPullingTime="2.540533012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:45:03.733053428 +0000 UTC m=+1566.396322652" watchObservedRunningTime="2026-07-21 08:45:03.734489311 +0000 UTC m=+1566.397758572" Jul 21 08:45:04.722696 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:04.722641 2569 generic.go:356] "Generic (PLEG): container finished" podID="fd30d57f-75b4-472b-a1c4-3e9c649e45a9" containerID="d73cf5e1a31b93d150467781cb31798a38489f15b59498ef62200cf494412f26" exitCode=0 Jul 21 08:45:04.723090 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:04.722745 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" event={"ID":"fd30d57f-75b4-472b-a1c4-3e9c649e45a9","Type":"ContainerDied","Data":"d73cf5e1a31b93d150467781cb31798a38489f15b59498ef62200cf494412f26"} Jul 21 08:45:05.865443 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:05.865418 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" Jul 21 08:45:06.734381 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:06.734353 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" Jul 21 08:45:06.734558 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:06.734342 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743725-fsb76" event={"ID":"fd30d57f-75b4-472b-a1c4-3e9c649e45a9","Type":"ContainerDied","Data":"adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a"} Jul 21 08:45:06.734558 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:45:06.734482 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adb81f3ec407a401daefb27f3ba85b38682f54a764c4b9cc2a09c048ad11398a" Jul 21 08:48:58.572691 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:48:58.572564 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:48:58.579170 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:48:58.575635 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:48:58.581202 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:48:58.581181 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:48:58.584350 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:48:58.584330 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:50:16.050603 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.050568 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v"] Jul 21 08:50:16.051120 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.050985 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd30d57f-75b4-472b-a1c4-3e9c649e45a9" containerName="cleanup" Jul 21 08:50:16.051120 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.050998 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd30d57f-75b4-472b-a1c4-3e9c649e45a9" containerName="cleanup" Jul 21 08:50:16.051120 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.051073 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd30d57f-75b4-472b-a1c4-3e9c649e45a9" containerName="cleanup" Jul 21 08:50:16.054114 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.054097 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:16.056785 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.056762 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:50:16.056897 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.056813 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:50:16.062605 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.062581 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v"] Jul 21 08:50:16.119843 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.119812 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v\" (UID: \"d76f711b-60a3-432c-b551-73ec310f044a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:16.220541 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.220509 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v\" (UID: \"d76f711b-60a3-432c-b551-73ec310f044a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:16.221176 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.221156 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v\" (UID: \"d76f711b-60a3-432c-b551-73ec310f044a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:16.366429 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.366339 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:16.500991 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:16.500967 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v"] Jul 21 08:50:16.502973 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:50:16.502946 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76f711b_60a3_432c_b551_73ec310f044a.slice/crio-1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c WatchSource:0}: Error finding container 1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c: Status 404 returned error can't find the container with id 1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c Jul 21 08:50:17.212564 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:17.212535 2569 generic.go:356] "Generic (PLEG): container finished" podID="d76f711b-60a3-432c-b551-73ec310f044a" containerID="dd4a22e376ac163b89bc371581d91db001426990972c93d9494347e9b50854df" exitCode=0 Jul 21 08:50:17.212951 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:17.212615 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" event={"ID":"d76f711b-60a3-432c-b551-73ec310f044a","Type":"ContainerDied","Data":"dd4a22e376ac163b89bc371581d91db001426990972c93d9494347e9b50854df"} Jul 21 08:50:17.212951 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:17.212672 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" event={"ID":"d76f711b-60a3-432c-b551-73ec310f044a","Type":"ContainerStarted","Data":"1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c"} Jul 21 08:50:18.350510 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:18.350485 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:18.441707 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:18.441642 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca\") pod \"d76f711b-60a3-432c-b551-73ec310f044a\" (UID: \"d76f711b-60a3-432c-b551-73ec310f044a\") " Jul 21 08:50:18.441978 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:18.441957 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d76f711b-60a3-432c-b551-73ec310f044a" (UID: "d76f711b-60a3-432c-b551-73ec310f044a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:50:18.542760 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:18.542730 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d76f711b-60a3-432c-b551-73ec310f044a-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:50:19.224163 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:19.224127 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" event={"ID":"d76f711b-60a3-432c-b551-73ec310f044a","Type":"ContainerDied","Data":"1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c"} Jul 21 08:50:19.224163 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:19.224164 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v" Jul 21 08:50:19.224381 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:19.224192 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f686542a76fa90efec907e4279e7332ac24cca83de3485143877abdd6426c4c" Jul 21 08:50:31.137121 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:31.137083 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v"] Jul 21 08:50:31.140185 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:31.140157 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-68712b6b-nhs9v"] Jul 21 08:50:31.834920 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:31.834886 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d76f711b-60a3-432c-b551-73ec310f044a" path="/var/lib/kubelet/pods/d76f711b-60a3-432c-b551-73ec310f044a/volumes" Jul 21 08:50:34.904470 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.904437 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:50:34.904904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.904829 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76f711b-60a3-432c-b551-73ec310f044a" containerName="revoke-keys" Jul 21 08:50:34.904904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.904841 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76f711b-60a3-432c-b551-73ec310f044a" containerName="revoke-keys" Jul 21 08:50:34.904904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.904898 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76f711b-60a3-432c-b551-73ec310f044a" containerName="revoke-keys" Jul 21 08:50:34.908018 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.907999 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:34.910745 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.910722 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-500e3541-serving-cert\"" Jul 21 08:50:34.910745 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.910738 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:50:34.910923 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.910739 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 08:50:34.911586 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.911566 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9jl5\"" Jul 21 08:50:34.919821 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:34.919790 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:50:35.097738 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.097699 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6w8k\" (UniqueName: \"kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.097738 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.097743 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.097950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.097856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.199324 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.199234 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6w8k\" (UniqueName: \"kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.199324 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.199272 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.199519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.199326 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.200057 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.199996 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.202079 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.202054 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.209095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.209069 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6w8k\" (UniqueName: \"kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k\") pod \"maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.220607 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.220585 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:35.367413 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.367380 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:50:35.369542 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:50:35.369511 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88498d62_20de_4c6e_8a14_161e6173332c.slice/crio-e4f5aeb2636e69e73da90327c0e4b42bb8f169083d5f4172caee24b63d2a7642 WatchSource:0}: Error finding container e4f5aeb2636e69e73da90327c0e4b42bb8f169083d5f4172caee24b63d2a7642: Status 404 returned error can't find the container with id e4f5aeb2636e69e73da90327c0e4b42bb8f169083d5f4172caee24b63d2a7642 Jul 21 08:50:35.372789 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:35.372765 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:50:36.311725 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:36.311652 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" event={"ID":"88498d62-20de-4c6e-8a14-161e6173332c","Type":"ContainerStarted","Data":"e4f5aeb2636e69e73da90327c0e4b42bb8f169083d5f4172caee24b63d2a7642"} Jul 21 08:50:38.323387 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:38.323339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" event={"ID":"88498d62-20de-4c6e-8a14-161e6173332c","Type":"ContainerStarted","Data":"1d1cca1deea4fb9bee3454e1cc77dd84a8c2b977b4936bba51e3de207c96a6d1"} Jul 21 08:50:38.323809 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:38.323418 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:38.344873 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:38.344816 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" podStartSLOduration=2.016356572 podStartE2EDuration="4.344800251s" totalImagesPullingTime="2.328443679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:50:38.342890379 +0000 UTC m=+1901.006159607" watchObservedRunningTime="2026-07-21 08:50:38.344800251 +0000 UTC m=+1901.008069493" Jul 21 08:50:44.334416 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:44.334387 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:50:59.861212 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:59.861179 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7"] Jul 21 08:50:59.864817 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:59.864801 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:50:59.868122 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:59.868100 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:50:59.876817 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:59.876791 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7"] Jul 21 08:50:59.932354 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:50:59.932317 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7\" (UID: \"8dc2fdbe-0678-4884-b398-8eccde7538e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:00.033182 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:00.033147 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7\" (UID: \"8dc2fdbe-0678-4884-b398-8eccde7538e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:00.033762 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:00.033742 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7\" (UID: \"8dc2fdbe-0678-4884-b398-8eccde7538e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:00.175308 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:00.175224 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:00.320253 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:00.320225 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7"] Jul 21 08:51:00.322258 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:51:00.322231 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc2fdbe_0678_4884_b398_8eccde7538e3.slice/crio-fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371 WatchSource:0}: Error finding container fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371: Status 404 returned error can't find the container with id fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371 Jul 21 08:51:00.428952 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:00.428866 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" event={"ID":"8dc2fdbe-0678-4884-b398-8eccde7538e3","Type":"ContainerStarted","Data":"fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371"} Jul 21 08:51:01.437779 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:01.437747 2569 generic.go:356] "Generic (PLEG): container finished" podID="8dc2fdbe-0678-4884-b398-8eccde7538e3" containerID="6ef58395d50e94496b955f2248a4d0cbe88e7dba0b270993521ea1b3634d93bc" exitCode=0 Jul 21 08:51:01.438195 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:01.437835 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" event={"ID":"8dc2fdbe-0678-4884-b398-8eccde7538e3","Type":"ContainerDied","Data":"6ef58395d50e94496b955f2248a4d0cbe88e7dba0b270993521ea1b3634d93bc"} Jul 21 08:51:02.577357 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:02.577330 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:02.657965 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:02.657928 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca\") pod \"8dc2fdbe-0678-4884-b398-8eccde7538e3\" (UID: \"8dc2fdbe-0678-4884-b398-8eccde7538e3\") " Jul 21 08:51:02.658289 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:02.658263 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8dc2fdbe-0678-4884-b398-8eccde7538e3" (UID: "8dc2fdbe-0678-4884-b398-8eccde7538e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:51:02.759501 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:02.759471 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dc2fdbe-0678-4884-b398-8eccde7538e3-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:03.448933 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:03.448893 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" event={"ID":"8dc2fdbe-0678-4884-b398-8eccde7538e3","Type":"ContainerDied","Data":"fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371"} Jul 21 08:51:03.448933 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:03.448912 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7" Jul 21 08:51:03.449151 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:03.448946 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd8529b9ec9e2b6a56b9b54daaf958e0a08c3162a76969fb65048b5b5bc16371" Jul 21 08:51:10.327108 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.327076 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:51:10.327488 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.327279 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" podUID="88498d62-20de-4c6e-8a14-161e6173332c" containerName="maas-api" containerID="cri-o://1d1cca1deea4fb9bee3454e1cc77dd84a8c2b977b4936bba51e3de207c96a6d1" gracePeriod=30 Jul 21 08:51:10.492063 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.492026 2569 generic.go:356] "Generic (PLEG): container finished" podID="88498d62-20de-4c6e-8a14-161e6173332c" containerID="1d1cca1deea4fb9bee3454e1cc77dd84a8c2b977b4936bba51e3de207c96a6d1" exitCode=0 Jul 21 08:51:10.492212 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.492085 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" event={"ID":"88498d62-20de-4c6e-8a14-161e6173332c","Type":"ContainerDied","Data":"1d1cca1deea4fb9bee3454e1cc77dd84a8c2b977b4936bba51e3de207c96a6d1"} Jul 21 08:51:10.579903 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.579839 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:51:10.732119 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.732081 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls\") pod \"88498d62-20de-4c6e-8a14-161e6173332c\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " Jul 21 08:51:10.732306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.732155 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6w8k\" (UniqueName: \"kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k\") pod \"88498d62-20de-4c6e-8a14-161e6173332c\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " Jul 21 08:51:10.732306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.732177 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca\") pod \"88498d62-20de-4c6e-8a14-161e6173332c\" (UID: \"88498d62-20de-4c6e-8a14-161e6173332c\") " Jul 21 08:51:10.732557 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.732526 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88498d62-20de-4c6e-8a14-161e6173332c" (UID: "88498d62-20de-4c6e-8a14-161e6173332c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:51:10.734583 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.734560 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88498d62-20de-4c6e-8a14-161e6173332c" (UID: "88498d62-20de-4c6e-8a14-161e6173332c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:51:10.734669 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.734583 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k" (OuterVolumeSpecName: "kube-api-access-k6w8k") pod "88498d62-20de-4c6e-8a14-161e6173332c" (UID: "88498d62-20de-4c6e-8a14-161e6173332c"). InnerVolumeSpecName "kube-api-access-k6w8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:51:10.833823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.833738 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88498d62-20de-4c6e-8a14-161e6173332c-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:10.833823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.833766 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6w8k\" (UniqueName: \"kubernetes.io/projected/88498d62-20de-4c6e-8a14-161e6173332c-kube-api-access-k6w8k\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:10.833823 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:10.833778 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88498d62-20de-4c6e-8a14-161e6173332c-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:11.499099 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.499069 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" Jul 21 08:51:11.499519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.499105 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r" event={"ID":"88498d62-20de-4c6e-8a14-161e6173332c","Type":"ContainerDied","Data":"e4f5aeb2636e69e73da90327c0e4b42bb8f169083d5f4172caee24b63d2a7642"} Jul 21 08:51:11.499519 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.499173 2569 scope.go:117] "RemoveContainer" containerID="1d1cca1deea4fb9bee3454e1cc77dd84a8c2b977b4936bba51e3de207c96a6d1" Jul 21 08:51:11.521709 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.521667 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:51:11.527456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.527432 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-500e3541-69ddc7d6c9-b6g5r"] Jul 21 08:51:11.834038 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:11.833957 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88498d62-20de-4c6e-8a14-161e6173332c" path="/var/lib/kubelet/pods/88498d62-20de-4c6e-8a14-161e6173332c/volumes" Jul 21 08:51:14.941241 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:14.941205 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7"] Jul 21 08:51:14.944631 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:14.944604 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-500e3541-vcdx7"] Jul 21 08:51:15.835248 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:15.835211 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dc2fdbe-0678-4884-b398-8eccde7538e3" path="/var/lib/kubelet/pods/8dc2fdbe-0678-4884-b398-8eccde7538e3/volumes" Jul 21 08:51:18.157701 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.157656 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:18.158164 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158145 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88498d62-20de-4c6e-8a14-161e6173332c" containerName="maas-api" Jul 21 08:51:18.158164 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158162 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="88498d62-20de-4c6e-8a14-161e6173332c" containerName="maas-api" Jul 21 08:51:18.158269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158174 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dc2fdbe-0678-4884-b398-8eccde7538e3" containerName="revoke-keys" Jul 21 08:51:18.158269 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158181 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dc2fdbe-0678-4884-b398-8eccde7538e3" containerName="revoke-keys" Jul 21 08:51:18.158369 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158284 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="88498d62-20de-4c6e-8a14-161e6173332c" containerName="maas-api" Jul 21 08:51:18.158369 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.158295 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dc2fdbe-0678-4884-b398-8eccde7538e3" containerName="revoke-keys" Jul 21 08:51:18.162470 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.162450 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.165108 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.165082 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 08:51:18.165108 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.165102 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-81c79699-serving-cert\"" Jul 21 08:51:18.166793 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.166771 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9jl5\"" Jul 21 08:51:18.166908 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.166816 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:51:18.172310 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.172287 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:18.297486 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.297460 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.297726 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.297525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.297726 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.297591 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzcvd\" (UniqueName: \"kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.398274 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.398229 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.398449 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.398286 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzcvd\" (UniqueName: \"kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.398449 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.398335 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.398922 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.398895 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.400901 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.400875 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.418586 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.418527 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzcvd\" (UniqueName: \"kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd\") pod \"maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.475621 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.475587 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:18.636361 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:18.636326 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:18.639938 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:51:18.639911 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53bc9bdc_b8ed_431a_9c67_6fe7ba835233.slice/crio-cf224d1207f814f68d644d07416c2aa3ca89139441cb20da906a04f264585279 WatchSource:0}: Error finding container cf224d1207f814f68d644d07416c2aa3ca89139441cb20da906a04f264585279: Status 404 returned error can't find the container with id cf224d1207f814f68d644d07416c2aa3ca89139441cb20da906a04f264585279 Jul 21 08:51:19.542871 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:19.542824 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" event={"ID":"53bc9bdc-b8ed-431a-9c67-6fe7ba835233","Type":"ContainerStarted","Data":"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e"} Jul 21 08:51:19.543380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:19.542887 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" event={"ID":"53bc9bdc-b8ed-431a-9c67-6fe7ba835233","Type":"ContainerStarted","Data":"cf224d1207f814f68d644d07416c2aa3ca89139441cb20da906a04f264585279"} Jul 21 08:51:19.543380 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:19.542975 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:19.579495 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:19.579432 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" podStartSLOduration=1.190493849 podStartE2EDuration="1.579411798s" totalImagesPullingTime="388.917949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:51:19.572905155 +0000 UTC m=+1942.236174382" watchObservedRunningTime="2026-07-21 08:51:19.579411798 +0000 UTC m=+1942.242681024" Jul 21 08:51:25.553253 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:25.553227 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:43.834775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:43.834740 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms"] Jul 21 08:51:43.838222 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:43.838205 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:43.841533 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:43.841499 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:51:43.843798 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:43.843769 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms"] Jul 21 08:51:43.930537 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:43.930501 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81c79699-8rrms\" (UID: \"c5b5ca21-bf83-4755-a221-adabda3378cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:44.032065 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.032025 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81c79699-8rrms\" (UID: \"c5b5ca21-bf83-4755-a221-adabda3378cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:44.032658 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.032638 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-81c79699-8rrms\" (UID: \"c5b5ca21-bf83-4755-a221-adabda3378cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:44.151044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.150946 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:44.287661 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.287634 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms"] Jul 21 08:51:44.290069 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:51:44.290040 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b5ca21_bf83_4755_a221_adabda3378cc.slice/crio-ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb WatchSource:0}: Error finding container ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb: Status 404 returned error can't find the container with id ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb Jul 21 08:51:44.663518 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.663483 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" event={"ID":"c5b5ca21-bf83-4755-a221-adabda3378cc","Type":"ContainerStarted","Data":"72b7940259034576d587c95f663d3e49d4e9e63bc45ddc3f9f282a7af79c37b5"} Jul 21 08:51:44.663737 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.663535 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" event={"ID":"c5b5ca21-bf83-4755-a221-adabda3378cc","Type":"ContainerStarted","Data":"ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb"} Jul 21 08:51:44.685794 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:44.685742 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" podStartSLOduration=1.685726896 podStartE2EDuration="1.685726896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:51:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:51:44.683664812 +0000 UTC m=+1967.346934041" watchObservedRunningTime="2026-07-21 08:51:44.685726896 +0000 UTC m=+1967.348996120" Jul 21 08:51:45.669834 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:45.669795 2569 generic.go:356] "Generic (PLEG): container finished" podID="c5b5ca21-bf83-4755-a221-adabda3378cc" containerID="72b7940259034576d587c95f663d3e49d4e9e63bc45ddc3f9f282a7af79c37b5" exitCode=0 Jul 21 08:51:45.670239 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:45.669867 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" event={"ID":"c5b5ca21-bf83-4755-a221-adabda3378cc","Type":"ContainerDied","Data":"72b7940259034576d587c95f663d3e49d4e9e63bc45ddc3f9f282a7af79c37b5"} Jul 21 08:51:46.810387 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:46.810358 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:46.863884 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:46.863850 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca\") pod \"c5b5ca21-bf83-4755-a221-adabda3378cc\" (UID: \"c5b5ca21-bf83-4755-a221-adabda3378cc\") " Jul 21 08:51:46.864191 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:46.864167 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5b5ca21-bf83-4755-a221-adabda3378cc" (UID: "c5b5ca21-bf83-4755-a221-adabda3378cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:51:46.965642 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:46.965534 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b5ca21-bf83-4755-a221-adabda3378cc-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:47.680790 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:47.680759 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" Jul 21 08:51:47.680790 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:47.680774 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms" event={"ID":"c5b5ca21-bf83-4755-a221-adabda3378cc","Type":"ContainerDied","Data":"ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb"} Jul 21 08:51:47.681070 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:47.680825 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed93012e03a3123fd40fe7513f4b5a99e78b9420ebc7571de02cbb9c34f104cb" Jul 21 08:51:54.327904 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.327869 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:54.328736 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.328137 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" podUID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" containerName="maas-api" containerID="cri-o://ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e" gracePeriod=30 Jul 21 08:51:54.584306 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.584241 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:54.716235 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.716195 2569 generic.go:356] "Generic (PLEG): container finished" podID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" containerID="ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e" exitCode=0 Jul 21 08:51:54.716434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.716242 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" Jul 21 08:51:54.716434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.716282 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" event={"ID":"53bc9bdc-b8ed-431a-9c67-6fe7ba835233","Type":"ContainerDied","Data":"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e"} Jul 21 08:51:54.716434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.716339 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7" event={"ID":"53bc9bdc-b8ed-431a-9c67-6fe7ba835233","Type":"ContainerDied","Data":"cf224d1207f814f68d644d07416c2aa3ca89139441cb20da906a04f264585279"} Jul 21 08:51:54.716434 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.716366 2569 scope.go:117] "RemoveContainer" containerID="ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e" Jul 21 08:51:54.725278 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.725257 2569 scope.go:117] "RemoveContainer" containerID="ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e" Jul 21 08:51:54.725556 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:51:54.725536 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e\": container with ID starting with ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e not found: ID does not exist" containerID="ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e" Jul 21 08:51:54.725647 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.725566 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e"} err="failed to get container status \"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e\": rpc error: code = NotFound desc = could not find container \"ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e\": container with ID starting with ed8f98072586d170126a40539204e1bbfbf2fa2972f2ebdef21c68157eee022e not found: ID does not exist" Jul 21 08:51:54.735932 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.735910 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca\") pod \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " Jul 21 08:51:54.736000 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.735975 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls\") pod \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " Jul 21 08:51:54.736051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.736037 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzcvd\" (UniqueName: \"kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd\") pod \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\" (UID: \"53bc9bdc-b8ed-431a-9c67-6fe7ba835233\") " Jul 21 08:51:54.736275 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.736253 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53bc9bdc-b8ed-431a-9c67-6fe7ba835233" (UID: "53bc9bdc-b8ed-431a-9c67-6fe7ba835233"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:51:54.736357 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.736337 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:54.738227 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.738197 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd" (OuterVolumeSpecName: "kube-api-access-xzcvd") pod "53bc9bdc-b8ed-431a-9c67-6fe7ba835233" (UID: "53bc9bdc-b8ed-431a-9c67-6fe7ba835233"). InnerVolumeSpecName "kube-api-access-xzcvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:51:54.738329 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.738237 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53bc9bdc-b8ed-431a-9c67-6fe7ba835233" (UID: "53bc9bdc-b8ed-431a-9c67-6fe7ba835233"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:51:54.837106 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.837008 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:54.837106 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:54.837051 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzcvd\" (UniqueName: \"kubernetes.io/projected/53bc9bdc-b8ed-431a-9c67-6fe7ba835233-kube-api-access-xzcvd\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:51:55.041521 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:55.041485 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:55.045531 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:55.045500 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-81c79699-79c9c48cff-pc9x7"] Jul 21 08:51:55.834931 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:55.834901 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" path="/var/lib/kubelet/pods/53bc9bdc-b8ed-431a-9c67-6fe7ba835233/volumes" Jul 21 08:51:58.909454 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:58.909410 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms"] Jul 21 08:51:58.911593 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:58.911563 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-81c79699-8rrms"] Jul 21 08:51:59.835609 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:51:59.835578 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b5ca21-bf83-4755-a221-adabda3378cc" path="/var/lib/kubelet/pods/c5b5ca21-bf83-4755-a221-adabda3378cc/volumes" Jul 21 08:52:03.381458 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381423 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:03.381877 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381829 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" containerName="maas-api" Jul 21 08:52:03.381877 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381842 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" containerName="maas-api" Jul 21 08:52:03.381877 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381863 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b5ca21-bf83-4755-a221-adabda3378cc" containerName="revoke-keys" Jul 21 08:52:03.381877 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381869 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b5ca21-bf83-4755-a221-adabda3378cc" containerName="revoke-keys" Jul 21 08:52:03.382007 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381929 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="53bc9bdc-b8ed-431a-9c67-6fe7ba835233" containerName="maas-api" Jul 21 08:52:03.382007 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.381936 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b5ca21-bf83-4755-a221-adabda3378cc" containerName="revoke-keys" Jul 21 08:52:03.384876 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.384859 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.388575 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.388554 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9jl5\"" Jul 21 08:52:03.389139 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.389116 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-84fe4740-serving-cert\"" Jul 21 08:52:03.389520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.389507 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:52:03.389763 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.389748 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 08:52:03.409093 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.409068 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:03.518752 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.518713 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.518942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.518836 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.518942 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.518881 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvwb7\" (UniqueName: \"kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.619787 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.619748 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.619988 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.619856 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.619988 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.619905 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvwb7\" (UniqueName: \"kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.620397 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.620374 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.622532 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.622509 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.631342 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.631320 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvwb7\" (UniqueName: \"kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7\") pod \"maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.696603 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.696490 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:03.852362 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:03.852328 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:03.853476 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:52:03.853445 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a40bf6e_d02a_490b_a657_77951aaadde3.slice/crio-1d78eadd06b08071c60bdebf1095152842d20f2c527decedabe8f470bdb3744a WatchSource:0}: Error finding container 1d78eadd06b08071c60bdebf1095152842d20f2c527decedabe8f470bdb3744a: Status 404 returned error can't find the container with id 1d78eadd06b08071c60bdebf1095152842d20f2c527decedabe8f470bdb3744a Jul 21 08:52:04.782049 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:04.782013 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" event={"ID":"0a40bf6e-d02a-490b-a657-77951aaadde3","Type":"ContainerStarted","Data":"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c"} Jul 21 08:52:04.782445 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:04.782069 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" event={"ID":"0a40bf6e-d02a-490b-a657-77951aaadde3","Type":"ContainerStarted","Data":"1d78eadd06b08071c60bdebf1095152842d20f2c527decedabe8f470bdb3744a"} Jul 21 08:52:04.782445 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:04.782156 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:04.807888 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:04.807837 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" podStartSLOduration=1.267263846 podStartE2EDuration="1.807820186s" totalImagesPullingTime="540.55634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:52:04.8065029 +0000 UTC m=+1987.469772121" watchObservedRunningTime="2026-07-21 08:52:04.807820186 +0000 UTC m=+1987.471089411" Jul 21 08:52:10.792656 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:10.792629 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:28.112789 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.112757 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz"] Jul 21 08:52:28.116396 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.116380 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:28.119798 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.119774 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:52:28.129802 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.129769 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz"] Jul 21 08:52:28.252452 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.252411 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz\" (UID: \"a2c69295-afde-47e6-a45b-cf60b00e1a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:28.353474 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.353439 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz\" (UID: \"a2c69295-afde-47e6-a45b-cf60b00e1a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:28.354135 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.354115 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz\" (UID: \"a2c69295-afde-47e6-a45b-cf60b00e1a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:28.427236 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.427148 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:28.562664 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.562633 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz"] Jul 21 08:52:28.900520 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.900481 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" event={"ID":"a2c69295-afde-47e6-a45b-cf60b00e1a9e","Type":"ContainerStarted","Data":"df3bf481b16a59ea8f793ad60fea1021614aad8b9c8eafbb9d6a595bd55b4456"} Jul 21 08:52:28.900715 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.900542 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" event={"ID":"a2c69295-afde-47e6-a45b-cf60b00e1a9e","Type":"ContainerStarted","Data":"a520adcef938d3048f1640c341ef262df7053211d124fd92aead6220e1aafc48"} Jul 21 08:52:28.917963 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:28.917914 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" podStartSLOduration=0.917898919 podStartE2EDuration="917.898919ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:52:28.917633789 +0000 UTC m=+2011.580903015" watchObservedRunningTime="2026-07-21 08:52:28.917898919 +0000 UTC m=+2011.581168143" Jul 21 08:52:29.906750 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:29.906715 2569 generic.go:356] "Generic (PLEG): container finished" podID="a2c69295-afde-47e6-a45b-cf60b00e1a9e" containerID="df3bf481b16a59ea8f793ad60fea1021614aad8b9c8eafbb9d6a595bd55b4456" exitCode=0 Jul 21 08:52:29.907215 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:29.906775 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" event={"ID":"a2c69295-afde-47e6-a45b-cf60b00e1a9e","Type":"ContainerDied","Data":"df3bf481b16a59ea8f793ad60fea1021614aad8b9c8eafbb9d6a595bd55b4456"} Jul 21 08:52:31.048931 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.048909 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:31.178836 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.178738 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca\") pod \"a2c69295-afde-47e6-a45b-cf60b00e1a9e\" (UID: \"a2c69295-afde-47e6-a45b-cf60b00e1a9e\") " Jul 21 08:52:31.179116 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.179091 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2c69295-afde-47e6-a45b-cf60b00e1a9e" (UID: "a2c69295-afde-47e6-a45b-cf60b00e1a9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:52:31.280051 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.280015 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2c69295-afde-47e6-a45b-cf60b00e1a9e-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:52:31.919004 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.918969 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" event={"ID":"a2c69295-afde-47e6-a45b-cf60b00e1a9e","Type":"ContainerDied","Data":"a520adcef938d3048f1640c341ef262df7053211d124fd92aead6220e1aafc48"} Jul 21 08:52:31.919193 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.919023 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a520adcef938d3048f1640c341ef262df7053211d124fd92aead6220e1aafc48" Jul 21 08:52:31.919193 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:31.918977 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz" Jul 21 08:52:39.166429 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.166395 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:39.166897 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.166604 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" podUID="0a40bf6e-d02a-490b-a657-77951aaadde3" containerName="maas-api" containerID="cri-o://b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c" gracePeriod=30 Jul 21 08:52:39.413916 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.413889 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:39.563776 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.563736 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls\") pod \"0a40bf6e-d02a-490b-a657-77951aaadde3\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " Jul 21 08:52:39.563962 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.563836 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvwb7\" (UniqueName: \"kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7\") pod \"0a40bf6e-d02a-490b-a657-77951aaadde3\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " Jul 21 08:52:39.563962 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.563886 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca\") pod \"0a40bf6e-d02a-490b-a657-77951aaadde3\" (UID: \"0a40bf6e-d02a-490b-a657-77951aaadde3\") " Jul 21 08:52:39.564278 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.564250 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a40bf6e-d02a-490b-a657-77951aaadde3" (UID: "0a40bf6e-d02a-490b-a657-77951aaadde3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:52:39.566132 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.566095 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7" (OuterVolumeSpecName: "kube-api-access-tvwb7") pod "0a40bf6e-d02a-490b-a657-77951aaadde3" (UID: "0a40bf6e-d02a-490b-a657-77951aaadde3"). InnerVolumeSpecName "kube-api-access-tvwb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:52:39.566261 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.566101 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a40bf6e-d02a-490b-a657-77951aaadde3" (UID: "0a40bf6e-d02a-490b-a657-77951aaadde3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:52:39.665321 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.665283 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvwb7\" (UniqueName: \"kubernetes.io/projected/0a40bf6e-d02a-490b-a657-77951aaadde3-kube-api-access-tvwb7\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:52:39.665321 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.665315 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a40bf6e-d02a-490b-a657-77951aaadde3-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:52:39.665321 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.665327 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a40bf6e-d02a-490b-a657-77951aaadde3-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:52:39.958870 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.958831 2569 generic.go:356] "Generic (PLEG): container finished" podID="0a40bf6e-d02a-490b-a657-77951aaadde3" containerID="b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c" exitCode=0 Jul 21 08:52:39.959044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.958880 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" Jul 21 08:52:39.959044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.958913 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" event={"ID":"0a40bf6e-d02a-490b-a657-77951aaadde3","Type":"ContainerDied","Data":"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c"} Jul 21 08:52:39.959044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.958970 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz" event={"ID":"0a40bf6e-d02a-490b-a657-77951aaadde3","Type":"ContainerDied","Data":"1d78eadd06b08071c60bdebf1095152842d20f2c527decedabe8f470bdb3744a"} Jul 21 08:52:39.959044 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.958993 2569 scope.go:117] "RemoveContainer" containerID="b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c" Jul 21 08:52:39.968067 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.968044 2569 scope.go:117] "RemoveContainer" containerID="b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c" Jul 21 08:52:39.968358 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:52:39.968334 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c\": container with ID starting with b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c not found: ID does not exist" containerID="b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c" Jul 21 08:52:39.968410 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.968370 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c"} err="failed to get container status \"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c\": rpc error: code = NotFound desc = could not find container \"b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c\": container with ID starting with b32b25a045f20f4a8a1e8f84383b0eaeb061a362728001c118a09b310902090c not found: ID does not exist" Jul 21 08:52:39.976633 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.976607 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:39.980616 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:39.980592 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-84fe4740-78c88bc7c6-dmdhz"] Jul 21 08:52:41.834786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:41.834752 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a40bf6e-d02a-490b-a657-77951aaadde3" path="/var/lib/kubelet/pods/0a40bf6e-d02a-490b-a657-77951aaadde3/volumes" Jul 21 08:52:43.182729 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:43.182693 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz"] Jul 21 08:52:43.185460 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:43.185427 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-84fe4740-rd9tz"] Jul 21 08:52:43.835057 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:52:43.835025 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c69295-afde-47e6-a45b-cf60b00e1a9e" path="/var/lib/kubelet/pods/a2c69295-afde-47e6-a45b-cf60b00e1a9e/volumes" Jul 21 08:53:58.603477 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:53:58.603362 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:53:58.607753 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:53:58.607732 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:53:58.612228 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:53:58.612206 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:53:58.615766 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:53:58.615748 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:54:28.601542 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601510 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601902 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c69295-afde-47e6-a45b-cf60b00e1a9e" containerName="revoke-keys" Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601915 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c69295-afde-47e6-a45b-cf60b00e1a9e" containerName="revoke-keys" Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601934 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a40bf6e-d02a-490b-a657-77951aaadde3" containerName="maas-api" Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601940 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a40bf6e-d02a-490b-a657-77951aaadde3" containerName="maas-api" Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.601993 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a40bf6e-d02a-490b-a657-77951aaadde3" containerName="maas-api" Jul 21 08:54:28.602023 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.602006 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c69295-afde-47e6-a45b-cf60b00e1a9e" containerName="revoke-keys" Jul 21 08:54:28.605131 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.605110 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.607791 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.607766 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9jl5\"" Jul 21 08:54:28.608063 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.608035 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:54:28.608628 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.608607 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c2dd45b4-serving-cert\"" Jul 21 08:54:28.608750 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.608614 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 08:54:28.619040 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.619013 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:28.649980 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.649939 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.650150 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.649998 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj4s6\" (UniqueName: \"kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.650150 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.650049 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.750905 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.750872 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj4s6\" (UniqueName: \"kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.751095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.750945 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.751095 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.751038 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.751641 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.751618 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.753437 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.753418 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.762550 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.762524 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj4s6\" (UniqueName: \"kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6\") pod \"maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:28.916798 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:28.916698 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:29.065481 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:29.065452 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:29.512927 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:29.512890 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" event={"ID":"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb","Type":"ContainerStarted","Data":"13df7d097f0a00d0cf2a1f1e1d30f33e1db6f5f4cd5a10c27bdbaf9d88a4a9cc"} Jul 21 08:54:30.519775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:30.519728 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" event={"ID":"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb","Type":"ContainerStarted","Data":"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a"} Jul 21 08:54:30.520208 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:30.519926 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:30.543950 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:30.543892 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" podStartSLOduration=2.065766496 podStartE2EDuration="2.543872222s" totalImagesPullingTime="478.105726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:54:30.542195989 +0000 UTC m=+2133.205465218" watchObservedRunningTime="2026-07-21 08:54:30.543872222 +0000 UTC m=+2133.207141446" Jul 21 08:54:36.529425 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:36.529396 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:40.423927 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.423885 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw"] Jul 21 08:54:40.427629 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.427605 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:40.430361 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.430334 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:54:40.436326 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.436297 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw"] Jul 21 08:54:40.560046 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.560003 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw\" (UID: \"2fbb05b9-9853-4187-8e37-cd14ec0e7a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:40.660945 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.660907 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw\" (UID: \"2fbb05b9-9853-4187-8e37-cd14ec0e7a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:40.661512 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.661493 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw\" (UID: \"2fbb05b9-9853-4187-8e37-cd14ec0e7a55\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:40.739444 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.739359 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:40.884396 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:40.884367 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw"] Jul 21 08:54:40.887777 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:54:40.887742 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fbb05b9_9853_4187_8e37_cd14ec0e7a55.slice/crio-6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73 WatchSource:0}: Error finding container 6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73: Status 404 returned error can't find the container with id 6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73 Jul 21 08:54:41.576816 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:41.576789 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" event={"ID":"2fbb05b9-9853-4187-8e37-cd14ec0e7a55","Type":"ContainerStarted","Data":"54a747b2518e49b051090ade16412d6658462d69c0173ae550bde03dc72b235d"} Jul 21 08:54:41.577104 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:41.576852 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" event={"ID":"2fbb05b9-9853-4187-8e37-cd14ec0e7a55","Type":"ContainerStarted","Data":"6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73"} Jul 21 08:54:42.583697 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.583649 2569 generic.go:356] "Generic (PLEG): container finished" podID="2fbb05b9-9853-4187-8e37-cd14ec0e7a55" containerID="54a747b2518e49b051090ade16412d6658462d69c0173ae550bde03dc72b235d" exitCode=0 Jul 21 08:54:42.584093 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.583750 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" event={"ID":"2fbb05b9-9853-4187-8e37-cd14ec0e7a55","Type":"ContainerDied","Data":"54a747b2518e49b051090ade16412d6658462d69c0173ae550bde03dc72b235d"} Jul 21 08:54:42.720234 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.720211 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:42.779913 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.779871 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca\") pod \"2fbb05b9-9853-4187-8e37-cd14ec0e7a55\" (UID: \"2fbb05b9-9853-4187-8e37-cd14ec0e7a55\") " Jul 21 08:54:42.780243 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.780217 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fbb05b9-9853-4187-8e37-cd14ec0e7a55" (UID: "2fbb05b9-9853-4187-8e37-cd14ec0e7a55"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:54:42.881070 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:42.880966 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fbb05b9-9853-4187-8e37-cd14ec0e7a55-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:54:43.590047 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:43.590015 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" Jul 21 08:54:43.590047 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:43.590041 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw" event={"ID":"2fbb05b9-9853-4187-8e37-cd14ec0e7a55","Type":"ContainerDied","Data":"6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73"} Jul 21 08:54:43.590491 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:43.590084 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6159789afb227a0857b51b2d093f77050970a6efde1661ecf82543a717baba73" Jul 21 08:54:50.908304 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:50.908218 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:50.908847 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:50.908474 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" podUID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" containerName="maas-api" containerID="cri-o://9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a" gracePeriod=30 Jul 21 08:54:51.172647 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.172576 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:51.260007 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.259973 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls\") pod \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " Jul 21 08:54:51.260184 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.260036 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca\") pod \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " Jul 21 08:54:51.260244 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.260219 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj4s6\" (UniqueName: \"kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6\") pod \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\" (UID: \"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb\") " Jul 21 08:54:51.260349 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.260333 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" (UID: "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:54:51.260508 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.260493 2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-openshift-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:54:51.262307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.262267 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6" (OuterVolumeSpecName: "kube-api-access-cj4s6") pod "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" (UID: "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb"). InnerVolumeSpecName "kube-api-access-cj4s6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 08:54:51.262307 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.262299 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" (UID: "55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 08:54:51.361744 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.361663 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj4s6\" (UniqueName: \"kubernetes.io/projected/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-kube-api-access-cj4s6\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:54:51.361744 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.361737 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb-maas-api-tls\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:54:51.639742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.639673 2569 generic.go:356] "Generic (PLEG): container finished" podID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" containerID="9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a" exitCode=0 Jul 21 08:54:51.639742 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.639746 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" Jul 21 08:54:51.639978 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.639752 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" event={"ID":"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb","Type":"ContainerDied","Data":"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a"} Jul 21 08:54:51.639978 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.639831 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq" event={"ID":"55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb","Type":"ContainerDied","Data":"13df7d097f0a00d0cf2a1f1e1d30f33e1db6f5f4cd5a10c27bdbaf9d88a4a9cc"} Jul 21 08:54:51.639978 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.639854 2569 scope.go:117] "RemoveContainer" containerID="9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a" Jul 21 08:54:51.649525 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.649508 2569 scope.go:117] "RemoveContainer" containerID="9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a" Jul 21 08:54:51.649895 ip-10-0-142-97 kubenswrapper[2569]: E0721 08:54:51.649876 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a\": container with ID starting with 9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a not found: ID does not exist" containerID="9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a" Jul 21 08:54:51.649977 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.649904 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a"} err="failed to get container status \"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a\": rpc error: code = NotFound desc = could not find container \"9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a\": container with ID starting with 9d4b017cce4914aa9aaea1ec95d0f45119dbbf60f316b933d168a98337ea1b1a not found: ID does not exist" Jul 21 08:54:51.660746 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.660714 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:51.665254 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.665224 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c2dd45b4-5554f6fb54-9xmjq"] Jul 21 08:54:51.834879 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:51.834848 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" path="/var/lib/kubelet/pods/55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb/volumes" Jul 21 08:54:55.511281 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:55.511242 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw"] Jul 21 08:54:55.515707 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:55.515667 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c2dd45b4-twxmw"] Jul 21 08:54:55.834600 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:54:55.834520 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fbb05b9-9853-4187-8e37-cd14ec0e7a55" path="/var/lib/kubelet/pods/2fbb05b9-9853-4187-8e37-cd14ec0e7a55/volumes" Jul 21 08:56:23.617965 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.617885 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79"] Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618259 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fbb05b9-9853-4187-8e37-cd14ec0e7a55" containerName="revoke-keys" Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618269 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fbb05b9-9853-4187-8e37-cd14ec0e7a55" containerName="revoke-keys" Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618287 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" containerName="maas-api" Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618292 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" containerName="maas-api" Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618355 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ba2dd1-3fd5-42d9-ab3f-4fae923ed7cb" containerName="maas-api" Jul 21 08:56:23.618421 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.618362 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fbb05b9-9853-4187-8e37-cd14ec0e7a55" containerName="revoke-keys" Jul 21 08:56:23.621196 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.621176 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:23.623667 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.623649 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 08:56:23.623792 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.623710 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 08:56:23.630097 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.630068 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79"] Jul 21 08:56:23.709934 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.709892 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f491e220-kww79\" (UID: \"b709532d-c4ec-4686-b6ff-033c4248e713\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:23.810730 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.810661 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f491e220-kww79\" (UID: \"b709532d-c4ec-4686-b6ff-033c4248e713\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:23.811339 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.811318 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f491e220-kww79\" (UID: \"b709532d-c4ec-4686-b6ff-033c4248e713\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:23.932271 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:23.932191 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:24.065631 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:24.065605 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79"] Jul 21 08:56:24.068367 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:56:24.068339 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb709532d_c4ec_4686_b6ff_033c4248e713.slice/crio-739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d WatchSource:0}: Error finding container 739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d: Status 404 returned error can't find the container with id 739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d Jul 21 08:56:24.105859 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:24.105828 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" event={"ID":"b709532d-c4ec-4686-b6ff-033c4248e713","Type":"ContainerStarted","Data":"739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d"} Jul 21 08:56:25.112751 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:25.112699 2569 generic.go:356] "Generic (PLEG): container finished" podID="b709532d-c4ec-4686-b6ff-033c4248e713" containerID="8912cb6c683d1db8dcc358cadb1663990ae1d469e56357f271f6212e47855cd9" exitCode=0 Jul 21 08:56:25.113144 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:25.112776 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" event={"ID":"b709532d-c4ec-4686-b6ff-033c4248e713","Type":"ContainerDied","Data":"8912cb6c683d1db8dcc358cadb1663990ae1d469e56357f271f6212e47855cd9"} Jul 21 08:56:26.244951 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:26.244926 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:26.332171 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:26.332140 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca\") pod \"b709532d-c4ec-4686-b6ff-033c4248e713\" (UID: \"b709532d-c4ec-4686-b6ff-033c4248e713\") " Jul 21 08:56:26.332467 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:26.332446 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b709532d-c4ec-4686-b6ff-033c4248e713" (UID: "b709532d-c4ec-4686-b6ff-033c4248e713"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 08:56:26.433357 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:26.433281 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b709532d-c4ec-4686-b6ff-033c4248e713-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 08:56:27.123807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:27.123774 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" event={"ID":"b709532d-c4ec-4686-b6ff-033c4248e713","Type":"ContainerDied","Data":"739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d"} Jul 21 08:56:27.123807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:27.123791 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79" Jul 21 08:56:27.124006 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:27.123848 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="739f047a3b2b63c04372b7dca6523de7a547cf1c938cf38713af6e04324d560d" Jul 21 08:56:43.704370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:43.704337 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79"] Jul 21 08:56:43.707438 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:43.707411 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f491e220-kww79"] Jul 21 08:56:43.834092 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:43.834059 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b709532d-c4ec-4686-b6ff-033c4248e713" path="/var/lib/kubelet/pods/b709532d-c4ec-4686-b6ff-033c4248e713/volumes" Jul 21 08:56:57.982768 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:56:57.982741 2569 scope.go:117] "RemoveContainer" containerID="dd4a22e376ac163b89bc371581d91db001426990972c93d9494347e9b50854df" Jul 21 08:57:58.002842 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:57:58.002754 2569 scope.go:117] "RemoveContainer" containerID="6ef58395d50e94496b955f2248a4d0cbe88e7dba0b270993521ea1b3634d93bc" Jul 21 08:57:58.011279 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:57:58.011259 2569 scope.go:117] "RemoveContainer" containerID="72b7940259034576d587c95f663d3e49d4e9e63bc45ddc3f9f282a7af79c37b5" Jul 21 08:58:41.334315 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.334278 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 08:58:41.334836 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.334818 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b709532d-c4ec-4686-b6ff-033c4248e713" containerName="revoke-keys" Jul 21 08:58:41.334882 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.334841 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b709532d-c4ec-4686-b6ff-033c4248e713" containerName="revoke-keys" Jul 21 08:58:41.334976 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.334965 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b709532d-c4ec-4686-b6ff-033c4248e713" containerName="revoke-keys" Jul 21 08:58:41.338298 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.338278 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.342046 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.342023 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c941d0-openshift-default-dockercfg-t9j8f\"" Jul 21 08:58:41.394294 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.394262 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 08:58:41.453283 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453251 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453307 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453472 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453369 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453478 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453538 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf5nf\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453583 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453627 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453729 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.453807 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.453752 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554545 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554506 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554563 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554585 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554611 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554660 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554721 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.554786 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554750 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.554788 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf5nf\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555031 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555046 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555101 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555092 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555345 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555192 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555345 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555329 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.555614 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.555490 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.557260 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.557238 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.557376 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.557260 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.581825 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.581790 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.581988 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.581965 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf5nf\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf\") pod \"e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.648775 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.648658 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:41.804529 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.804505 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 08:58:41.806941 ip-10-0-142-97 kubenswrapper[2569]: W0721 08:58:41.806907 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49dc7dbf_007e_4f49_a688_f08888a9e0c6.slice/crio-b62f119b956c8aa03ebf0064f73eb712d353baae72c2b651884d5afb0fa9a38c WatchSource:0}: Error finding container b62f119b956c8aa03ebf0064f73eb712d353baae72c2b651884d5afb0fa9a38c: Status 404 returned error can't find the container with id b62f119b956c8aa03ebf0064f73eb712d353baae72c2b651884d5afb0fa9a38c Jul 21 08:58:41.808899 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:41.808883 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 08:58:42.803138 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:42.803102 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerStarted","Data":"b62f119b956c8aa03ebf0064f73eb712d353baae72c2b651884d5afb0fa9a38c"} Jul 21 08:58:44.083665 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:44.083627 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:44.083963 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:44.083736 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:44.083963 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:44.083768 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:44.815827 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:44.815782 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerStarted","Data":"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460"} Jul 21 08:58:44.842379 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:44.842322 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podStartSLOduration=1.567995975 podStartE2EDuration="3.842304982s" totalImagesPullingTime="2.274309007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 08:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 08:58:44.840515675 +0000 UTC m=+2387.503784912" watchObservedRunningTime="2026-07-21 08:58:44.842304982 +0000 UTC m=+2387.505574208" Jul 21 08:58:45.649248 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:45.649207 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:45.653997 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:45.653972 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:45.823788 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:45.823763 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:45.824733 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:45.824699 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:58:58.034828 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.034786 2569 scope.go:117] "RemoveContainer" containerID="df3bf481b16a59ea8f793ad60fea1021614aad8b9c8eafbb9d6a595bd55b4456" Jul 21 08:58:58.647475 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.647447 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:58:58.648278 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.648252 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 08:58:58.656163 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.656139 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:58:58.656854 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.656836 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 08:58:58.894035 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.893936 2569 generic.go:356] "Generic (PLEG): container finished" podID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerID="8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460" exitCode=0 Jul 21 08:58:58.894035 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.894009 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerDied","Data":"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460"} Jul 21 08:58:58.894430 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.894416 2569 scope.go:117] "RemoveContainer" containerID="8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460" Jul 21 08:58:58.895365 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.895333 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:58.895456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.895409 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:58.895456 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:58.895443 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 08:58:59.906198 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:58:59.906158 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerStarted","Data":"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187"} Jul 21 08:59:00.649625 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:00.649582 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:59:00.650613 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:00.650590 2569 patch_prober.go:28] interesting pod/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 21 08:59:00.650676 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:00.650642 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 21 08:59:00.824370 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:00.824323 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:59:01.649509 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:01.649471 2569 patch_prober.go:28] interesting pod/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 21 08:59:01.649935 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:01.649551 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 21 08:59:02.649896 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:02.649857 2569 patch_prober.go:28] interesting pod/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 21 08:59:02.650298 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:02.649932 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 21 08:59:03.649165 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:03.649125 2569 patch_prober.go:28] interesting pod/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 21 08:59:03.649352 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:03.649191 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 21 08:59:04.653998 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:04.653967 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 08:59:04.655058 ip-10-0-142-97 kubenswrapper[2569]: I0721 08:59:04.655039 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 09:00:00.134817 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.134784 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2"] Jul 21 09:00:00.138531 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.138482 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" Jul 21 09:00:00.138713 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.138621 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt"] Jul 21 09:00:00.141801 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.141782 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:00:00.142313 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.142295 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx"] Jul 21 09:00:00.142392 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.142379 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" Jul 21 09:00:00.146568 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.146442 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" Jul 21 09:00:00.148576 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.148552 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2"] Jul 21 09:00:00.151371 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.151348 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx"] Jul 21 09:00:00.152036 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.152017 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" Jul 21 09:00:00.161207 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.161053 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" Jul 21 09:00:00.167456 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.167423 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt"] Jul 21 09:00:00.194302 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.194273 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" Jul 21 09:00:00.326336 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.326300 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt"] Jul 21 09:00:00.327476 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:00:00.327417 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0436a0ac_533e_4934_92a7_1661cb1695b8.slice/crio-c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0 WatchSource:0}: Error finding container c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0: Status 404 returned error can't find the container with id c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0 Jul 21 09:00:00.353083 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.353055 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2"] Jul 21 09:00:00.358814 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:00:00.358789 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7384dd33_70be_4eed_bed7_e056a5b723a5.slice/crio-7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69 WatchSource:0}: Error finding container 7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69: Status 404 returned error can't find the container with id 7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69 Jul 21 09:00:00.379037 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:00.379011 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx"] Jul 21 09:00:00.381493 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:00:00.381464 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924a9391_6029_4055_a246_f4e02cf7a48a.slice/crio-37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256 WatchSource:0}: Error finding container 37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256: Status 404 returned error can't find the container with id 37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256 Jul 21 09:00:01.218405 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.218370 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" event={"ID":"7384dd33-70be-4eed-bed7-e056a5b723a5","Type":"ContainerStarted","Data":"e5fbb54c36fceb7919cfcff7498e9c1f9ae2ca27cbbd0acf84e54b06b7ec799b"} Jul 21 09:00:01.218853 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.218424 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" event={"ID":"7384dd33-70be-4eed-bed7-e056a5b723a5","Type":"ContainerStarted","Data":"7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69"} Jul 21 09:00:01.220024 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.220002 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" event={"ID":"0436a0ac-533e-4934-92a7-1661cb1695b8","Type":"ContainerStarted","Data":"9fbcf4efa1cc0d808eab311f42f520f415e9437772b54cf6b2e9bd1995788108"} Jul 21 09:00:01.220114 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.220047 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" event={"ID":"0436a0ac-533e-4934-92a7-1661cb1695b8","Type":"ContainerStarted","Data":"c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0"} Jul 21 09:00:01.221391 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.221364 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" event={"ID":"924a9391-6029-4055-a246-f4e02cf7a48a","Type":"ContainerStarted","Data":"15086a0b36ba49727d9cbc2256f0156fbe01d1d7e6a3270deed8ca9029bc5a55"} Jul 21 09:00:01.221493 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.221434 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" event={"ID":"924a9391-6029-4055-a246-f4e02cf7a48a","Type":"ContainerStarted","Data":"37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256"} Jul 21 09:00:01.234688 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.234642 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" podStartSLOduration=1.23462928 podStartE2EDuration="1.23462928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:00:01.232520152 +0000 UTC m=+2463.895789368" watchObservedRunningTime="2026-07-21 09:00:01.23462928 +0000 UTC m=+2463.897898504" Jul 21 09:00:01.246736 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.246666 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" podStartSLOduration=1.246649796 podStartE2EDuration="1.246649796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:00:01.24649304 +0000 UTC m=+2463.909762266" watchObservedRunningTime="2026-07-21 09:00:01.246649796 +0000 UTC m=+2463.909919024" Jul 21 09:00:01.266706 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:01.264020 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" podStartSLOduration=1.263997659 podStartE2EDuration="1.263997659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:00:01.259904977 +0000 UTC m=+2463.923174199" watchObservedRunningTime="2026-07-21 09:00:01.263997659 +0000 UTC m=+2463.927266885" Jul 21 09:00:02.228358 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.228320 2569 generic.go:356] "Generic (PLEG): container finished" podID="0436a0ac-533e-4934-92a7-1661cb1695b8" containerID="9fbcf4efa1cc0d808eab311f42f520f415e9437772b54cf6b2e9bd1995788108" exitCode=0 Jul 21 09:00:02.228857 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.228412 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" event={"ID":"0436a0ac-533e-4934-92a7-1661cb1695b8","Type":"ContainerDied","Data":"9fbcf4efa1cc0d808eab311f42f520f415e9437772b54cf6b2e9bd1995788108"} Jul 21 09:00:02.229934 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.229912 2569 generic.go:356] "Generic (PLEG): container finished" podID="924a9391-6029-4055-a246-f4e02cf7a48a" containerID="15086a0b36ba49727d9cbc2256f0156fbe01d1d7e6a3270deed8ca9029bc5a55" exitCode=0 Jul 21 09:00:02.230060 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.229992 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" event={"ID":"924a9391-6029-4055-a246-f4e02cf7a48a","Type":"ContainerDied","Data":"15086a0b36ba49727d9cbc2256f0156fbe01d1d7e6a3270deed8ca9029bc5a55"} Jul 21 09:00:02.231514 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.231495 2569 generic.go:356] "Generic (PLEG): container finished" podID="7384dd33-70be-4eed-bed7-e056a5b723a5" containerID="e5fbb54c36fceb7919cfcff7498e9c1f9ae2ca27cbbd0acf84e54b06b7ec799b" exitCode=0 Jul 21 09:00:02.231599 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:02.231556 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" event={"ID":"7384dd33-70be-4eed-bed7-e056a5b723a5","Type":"ContainerDied","Data":"e5fbb54c36fceb7919cfcff7498e9c1f9ae2ca27cbbd0acf84e54b06b7ec799b"} Jul 21 09:00:03.395668 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:03.395637 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" Jul 21 09:00:03.429461 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:03.429433 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" Jul 21 09:00:03.432626 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:03.432598 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" Jul 21 09:00:04.243208 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.243117 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" Jul 21 09:00:04.243208 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.243124 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt" event={"ID":"0436a0ac-533e-4934-92a7-1661cb1695b8","Type":"ContainerDied","Data":"c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0"} Jul 21 09:00:04.243208 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.243176 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5f474dc8162ba090dbdc135db3c6a7712293a49a340b26f78a1983ce324a9f0" Jul 21 09:00:04.247377 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.247347 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" event={"ID":"924a9391-6029-4055-a246-f4e02cf7a48a","Type":"ContainerDied","Data":"37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256"} Jul 21 09:00:04.247570 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.247416 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37dc3c5f77e3c18be28021511ee36d82c6bf1f1ee0783962fde0cf59a4929256" Jul 21 09:00:04.247570 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.247360 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743740-4b9mx" Jul 21 09:00:04.248924 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.248885 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" event={"ID":"7384dd33-70be-4eed-bed7-e056a5b723a5","Type":"ContainerDied","Data":"7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69"} Jul 21 09:00:04.248924 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.248913 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2" Jul 21 09:00:04.249100 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:04.248932 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eddd495438a452b64d75f5de830cd4c43c90e39b3c2e0e26246be6c2810ac69" Jul 21 09:00:17.615642 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.615605 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x"] Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.615993 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7384dd33-70be-4eed-bed7-e056a5b723a5" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616004 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7384dd33-70be-4eed-bed7-e056a5b723a5" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616020 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0436a0ac-533e-4934-92a7-1661cb1695b8" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616025 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0436a0ac-533e-4934-92a7-1661cb1695b8" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616035 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="924a9391-6029-4055-a246-f4e02cf7a48a" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616043 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="924a9391-6029-4055-a246-f4e02cf7a48a" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616105 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0436a0ac-533e-4934-92a7-1661cb1695b8" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616113 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="924a9391-6029-4055-a246-f4e02cf7a48a" containerName="cleanup" Jul 21 09:00:17.616158 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.616119 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7384dd33-70be-4eed-bed7-e056a5b723a5" containerName="cleanup" Jul 21 09:00:17.619429 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.619412 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:17.623288 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.623259 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:00:17.624433 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.624411 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:00:17.659629 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.659592 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x\" (UID: \"d82b14d8-a933-4573-bb5d-ad11bed79aa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:17.707987 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.707948 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x"] Jul 21 09:00:17.760206 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.760165 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x\" (UID: \"d82b14d8-a933-4573-bb5d-ad11bed79aa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:17.760822 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.760804 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x\" (UID: \"d82b14d8-a933-4573-bb5d-ad11bed79aa3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:17.929779 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:17.929664 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:18.072655 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:18.072625 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x"] Jul 21 09:00:18.075479 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:00:18.075437 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82b14d8_a933_4573_bb5d_ad11bed79aa3.slice/crio-545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9 WatchSource:0}: Error finding container 545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9: Status 404 returned error can't find the container with id 545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9 Jul 21 09:00:18.320386 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:18.320352 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" event={"ID":"d82b14d8-a933-4573-bb5d-ad11bed79aa3","Type":"ContainerStarted","Data":"545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9"} Jul 21 09:00:19.327215 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:19.327179 2569 generic.go:356] "Generic (PLEG): container finished" podID="d82b14d8-a933-4573-bb5d-ad11bed79aa3" containerID="584a10f100a3e23ee2e29ae6f8f74e3fc51d8d7a90c531252e31ecb3a2ae8e18" exitCode=0 Jul 21 09:00:19.327647 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:19.327235 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" event={"ID":"d82b14d8-a933-4573-bb5d-ad11bed79aa3","Type":"ContainerDied","Data":"584a10f100a3e23ee2e29ae6f8f74e3fc51d8d7a90c531252e31ecb3a2ae8e18"} Jul 21 09:00:20.483325 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:20.483300 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:20.587834 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:20.587749 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca\") pod \"d82b14d8-a933-4573-bb5d-ad11bed79aa3\" (UID: \"d82b14d8-a933-4573-bb5d-ad11bed79aa3\") " Jul 21 09:00:20.588092 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:20.588069 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d82b14d8-a933-4573-bb5d-ad11bed79aa3" (UID: "d82b14d8-a933-4573-bb5d-ad11bed79aa3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:00:20.689125 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:20.689093 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d82b14d8-a933-4573-bb5d-ad11bed79aa3-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:00:21.340085 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:21.340054 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" Jul 21 09:00:21.340262 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:21.340087 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x" event={"ID":"d82b14d8-a933-4573-bb5d-ad11bed79aa3","Type":"ContainerDied","Data":"545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9"} Jul 21 09:00:21.340262 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:21.340132 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="545f5f0e4857dfce2b3e3fbcc5937c69e902307d47cc213bd02b5672bc5781f9" Jul 21 09:00:28.205671 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:28.205635 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt"] Jul 21 09:00:28.213047 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:28.213012 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8d885f-29743740-5wcxt"] Jul 21 09:00:29.834355 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:29.834317 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0436a0ac-533e-4934-92a7-1661cb1695b8" path="/var/lib/kubelet/pods/0436a0ac-533e-4934-92a7-1661cb1695b8/volumes" Jul 21 09:00:32.804613 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:32.804578 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x"] Jul 21 09:00:32.817306 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:32.817274 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8d885f-t8p6x"] Jul 21 09:00:33.833906 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:33.833878 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d82b14d8-a933-4573-bb5d-ad11bed79aa3" path="/var/lib/kubelet/pods/d82b14d8-a933-4573-bb5d-ad11bed79aa3/volumes" Jul 21 09:00:39.879452 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.879416 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc"] Jul 21 09:00:39.879895 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.879812 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d82b14d8-a933-4573-bb5d-ad11bed79aa3" containerName="revoke-keys" Jul 21 09:00:39.879895 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.879824 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d82b14d8-a933-4573-bb5d-ad11bed79aa3" containerName="revoke-keys" Jul 21 09:00:39.879895 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.879891 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d82b14d8-a933-4573-bb5d-ad11bed79aa3" containerName="revoke-keys" Jul 21 09:00:39.882977 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.882957 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:39.885754 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.885726 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:00:39.885900 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.885773 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:00:39.891623 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.891594 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc"] Jul 21 09:00:39.973106 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:39.973072 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc\" (UID: \"4ec34eef-f029-491c-912c-6a4e3edbb9c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:40.074123 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:40.074090 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc\" (UID: \"4ec34eef-f029-491c-912c-6a4e3edbb9c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:40.074693 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:40.074659 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc\" (UID: \"4ec34eef-f029-491c-912c-6a4e3edbb9c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:40.194321 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:40.194236 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:40.358036 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:40.358007 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc"] Jul 21 09:00:40.359987 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:00:40.359956 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec34eef_f029_491c_912c_6a4e3edbb9c2.slice/crio-0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb WatchSource:0}: Error finding container 0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb: Status 404 returned error can't find the container with id 0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb Jul 21 09:00:40.443762 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:40.443727 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" event={"ID":"4ec34eef-f029-491c-912c-6a4e3edbb9c2","Type":"ContainerStarted","Data":"0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb"} Jul 21 09:00:41.451009 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:41.450914 2569 generic.go:356] "Generic (PLEG): container finished" podID="4ec34eef-f029-491c-912c-6a4e3edbb9c2" containerID="1b9900f97ddd779da5cc440b93406b0bea164b2b9ba7475263699a1ed8bada05" exitCode=0 Jul 21 09:00:41.451436 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:41.451001 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" event={"ID":"4ec34eef-f029-491c-912c-6a4e3edbb9c2","Type":"ContainerDied","Data":"1b9900f97ddd779da5cc440b93406b0bea164b2b9ba7475263699a1ed8bada05"} Jul 21 09:00:42.592255 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:42.592228 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:42.698018 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:42.697975 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca\") pod \"4ec34eef-f029-491c-912c-6a4e3edbb9c2\" (UID: \"4ec34eef-f029-491c-912c-6a4e3edbb9c2\") " Jul 21 09:00:42.698330 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:42.698308 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ec34eef-f029-491c-912c-6a4e3edbb9c2" (UID: "4ec34eef-f029-491c-912c-6a4e3edbb9c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:00:42.799612 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:42.799578 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ec34eef-f029-491c-912c-6a4e3edbb9c2-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:00:43.469445 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:43.469418 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" Jul 21 09:00:43.469651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:43.469413 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc" event={"ID":"4ec34eef-f029-491c-912c-6a4e3edbb9c2","Type":"ContainerDied","Data":"0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb"} Jul 21 09:00:43.469651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:43.469577 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0633bb1f64332ce601390f68ce17774b5a2efc9dcf516dc520964941a84246eb" Jul 21 09:00:50.480580 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:50.480492 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2"] Jul 21 09:00:50.487843 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:50.487814 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c941d0-29743740-kgll2"] Jul 21 09:00:51.834265 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:51.834229 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7384dd33-70be-4eed-bed7-e056a5b723a5" path="/var/lib/kubelet/pods/7384dd33-70be-4eed-bed7-e056a5b723a5/volumes" Jul 21 09:00:54.977581 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:54.977547 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc"] Jul 21 09:00:54.987815 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:54.987786 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c941d0-m6qzc"] Jul 21 09:00:55.835018 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:55.834988 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ec34eef-f029-491c-912c-6a4e3edbb9c2" path="/var/lib/kubelet/pods/4ec34eef-f029-491c-912c-6a4e3edbb9c2/volumes" Jul 21 09:00:58.063494 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:00:58.063455 2569 scope.go:117] "RemoveContainer" containerID="54a747b2518e49b051090ade16412d6658462d69c0173ae550bde03dc72b235d" Jul 21 09:01:02.008320 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:02.008284 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 09:01:02.008869 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:02.008602 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" containerID="cri-o://3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187" gracePeriod=30 Jul 21 09:01:04.656526 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:04.656481 2569 patch_prober.go:28] interesting pod/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 09:01:04.656920 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:04.656549 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 09:01:07.272486 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.272408 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 09:01:07.313264 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313230 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313423 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313278 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313423 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313309 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313423 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313368 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313423 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313394 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313435 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf5nf\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313465 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313505 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313515 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:01:07.313651 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313541 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo\") pod \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\" (UID: \"49dc7dbf-007e-4f49-a688-f08888a9e0c6\") " Jul 21 09:01:07.313966 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313652 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:01:07.313966 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.313839 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:01:07.314074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.314025 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.314074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.314044 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-credential-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.314074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.314058 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-workload-certs\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.314285 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.314260 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:01:07.314368 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.314345 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:01:07.316822 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.316794 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf" (OuterVolumeSpecName: "kube-api-access-gf5nf") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "kube-api-access-gf5nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:01:07.317046 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.316897 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:01:07.317046 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.316905 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:01:07.317139 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.317081 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49dc7dbf-007e-4f49-a688-f08888a9e0c6" (UID: "49dc7dbf-007e-4f49-a688-f08888a9e0c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 09:01:07.414837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414798 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-envoy\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.414837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414824 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-data\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.414837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414833 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istiod-ca-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.414837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414842 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf5nf\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-kube-api-access-gf5nf\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.414837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414852 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-token\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.415245 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.414860 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49dc7dbf-007e-4f49-a688-f08888a9e0c6-istio-podinfo\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:07.601032 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.600928 2569 generic.go:356] "Generic (PLEG): container finished" podID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerID="3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187" exitCode=0 Jul 21 09:01:07.601032 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.600992 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" Jul 21 09:01:07.601032 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.601014 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerDied","Data":"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187"} Jul 21 09:01:07.601253 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.601092 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx" event={"ID":"49dc7dbf-007e-4f49-a688-f08888a9e0c6","Type":"ContainerDied","Data":"b62f119b956c8aa03ebf0064f73eb712d353baae72c2b651884d5afb0fa9a38c"} Jul 21 09:01:07.601253 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.601115 2569 scope.go:117] "RemoveContainer" containerID="3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187" Jul 21 09:01:07.611272 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.611251 2569 scope.go:117] "RemoveContainer" containerID="8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460" Jul 21 09:01:07.619403 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.619384 2569 scope.go:117] "RemoveContainer" containerID="3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187" Jul 21 09:01:07.619693 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:01:07.619657 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187\": container with ID starting with 3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187 not found: ID does not exist" containerID="3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187" Jul 21 09:01:07.619772 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.619708 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187"} err="failed to get container status \"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187\": rpc error: code = NotFound desc = could not find container \"3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187\": container with ID starting with 3fa96f000c4d4ffffb51e69e5c0046dc8b3e3fc693c7502d76d3a43f5ed5b187 not found: ID does not exist" Jul 21 09:01:07.619772 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.619726 2569 scope.go:117] "RemoveContainer" containerID="8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460" Jul 21 09:01:07.619978 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:01:07.619961 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460\": container with ID starting with 8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460 not found: ID does not exist" containerID="8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460" Jul 21 09:01:07.620020 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.619985 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460"} err="failed to get container status \"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460\": rpc error: code = NotFound desc = could not find container \"8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460\": container with ID starting with 8facf0ebc73a62f4e83eb87f51bcb0dac35cd1289d577a5bb3cd9a976d7de460 not found: ID does not exist" Jul 21 09:01:07.629926 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.629903 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 09:01:07.633885 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.633863 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c941d0-openshift-default-6699f45dfc-6cvmx"] Jul 21 09:01:07.834259 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:07.834227 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" path="/var/lib/kubelet/pods/49dc7dbf-007e-4f49-a688-f08888a9e0c6/volumes" Jul 21 09:01:10.993367 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993331 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:01:10.993908 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993890 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec34eef-f029-491c-912c-6a4e3edbb9c2" containerName="revoke-keys" Jul 21 09:01:10.993997 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993912 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec34eef-f029-491c-912c-6a4e3edbb9c2" containerName="revoke-keys" Jul 21 09:01:10.993997 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993928 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.993997 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993936 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.993997 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993950 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.993997 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.993959 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.994272 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.994056 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ec34eef-f029-491c-912c-6a4e3edbb9c2" containerName="revoke-keys" Jul 21 09:01:10.994272 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.994069 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.994272 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.994083 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dc7dbf-007e-4f49-a688-f08888a9e0c6" containerName="istio-proxy" Jul 21 09:01:10.998720 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:10.998675 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.001331 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.001313 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-bb8bc8-openshift-default-dockercfg-srhkd\"" Jul 21 09:01:11.015367 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.015345 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:01:11.050475 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050664 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050482 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050664 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050664 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050571 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv42n\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050664 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050614 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050664 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050631 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050969 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050742 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050969 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050826 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.050969 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.050884 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152383 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152344 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152383 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152384 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152408 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152431 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152457 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv42n\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152494 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152517 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152550 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.152649 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.152609 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.153047 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.153015 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.153114 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.153057 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.153114 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.153091 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.153223 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.153147 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.153223 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.153211 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.155556 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.155532 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.155799 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.155779 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.162455 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.162420 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv42n\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.162806 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.162787 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token\") pod \"e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.309537 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.309497 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:11.461603 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.461492 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:01:11.463953 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:01:11.463922 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0e52bc_7618_4593_b135_fe69ca5fb916.slice/crio-8cb83f788701316f910cc3a779c9f05cbea67f1553e6092d412fef3a980df459 WatchSource:0}: Error finding container 8cb83f788701316f910cc3a779c9f05cbea67f1553e6092d412fef3a980df459: Status 404 returned error can't find the container with id 8cb83f788701316f910cc3a779c9f05cbea67f1553e6092d412fef3a980df459 Jul 21 09:01:11.466217 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.466186 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:11.466327 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.466259 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:11.466327 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.466292 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:11.624550 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.624462 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerStarted","Data":"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55"} Jul 21 09:01:11.624550 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.624523 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerStarted","Data":"8cb83f788701316f910cc3a779c9f05cbea67f1553e6092d412fef3a980df459"} Jul 21 09:01:11.647916 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:11.647850 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podStartSLOduration=1.64782578 podStartE2EDuration="1.64782578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:01:11.646412112 +0000 UTC m=+2534.309681342" watchObservedRunningTime="2026-07-21 09:01:11.64782578 +0000 UTC m=+2534.311095007" Jul 21 09:01:12.309748 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:12.309714 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:12.314741 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:12.314715 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:12.629798 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:12.629715 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:12.630868 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:12.630847 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:26.361141 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.361103 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv"] Jul 21 09:01:26.365128 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.365090 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:26.368382 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.368348 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:01:26.368707 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.368663 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:01:26.377474 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.377435 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv"] Jul 21 09:01:26.506034 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.505996 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b88480-m9bfv\" (UID: \"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:26.606962 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.606921 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b88480-m9bfv\" (UID: \"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:26.607641 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.607620 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b88480-m9bfv\" (UID: \"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:26.687178 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.687082 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:26.845096 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:26.845061 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv"] Jul 21 09:01:26.849348 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:01:26.849306 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1e4cc1_3c79_40cd_b5b3_29e7ec64ce93.slice/crio-02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624 WatchSource:0}: Error finding container 02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624: Status 404 returned error can't find the container with id 02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624 Jul 21 09:01:27.724618 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:27.724578 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" event={"ID":"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93","Type":"ContainerStarted","Data":"21ce7cfa523812b63875b9f9c1307021afe872a67ba1297627abfd629a1f0d6a"} Jul 21 09:01:27.725075 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:27.724653 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" event={"ID":"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93","Type":"ContainerStarted","Data":"02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624"} Jul 21 09:01:27.740219 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:27.740161 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" podStartSLOduration=1.740142452 podStartE2EDuration="1.740142452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:01:27.73952443 +0000 UTC m=+2550.402793656" watchObservedRunningTime="2026-07-21 09:01:27.740142452 +0000 UTC m=+2550.403411678" Jul 21 09:01:28.672874 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:28.672832 2569 patch_prober.go:28] interesting pod/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 09:01:28.673064 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:28.672911 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 09:01:28.731349 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:28.731310 2569 generic.go:356] "Generic (PLEG): container finished" podID="0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" containerID="21ce7cfa523812b63875b9f9c1307021afe872a67ba1297627abfd629a1f0d6a" exitCode=0 Jul 21 09:01:28.731814 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:28.731390 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" event={"ID":"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93","Type":"ContainerDied","Data":"21ce7cfa523812b63875b9f9c1307021afe872a67ba1297627abfd629a1f0d6a"} Jul 21 09:01:29.738478 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.738379 2569 generic.go:356] "Generic (PLEG): container finished" podID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerID="a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55" exitCode=0 Jul 21 09:01:29.738478 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.738449 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerDied","Data":"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55"} Jul 21 09:01:29.739038 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.738985 2569 scope.go:117] "RemoveContainer" containerID="a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55" Jul 21 09:01:29.739991 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.739914 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:29.739991 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.740001 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:29.739991 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.740033 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:01:29.902867 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:29.902841 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:30.040604 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.040576 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca\") pod \"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93\" (UID: \"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93\") " Jul 21 09:01:30.040955 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.040932 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" (UID: "0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:01:30.141846 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.141807 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:30.745294 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.745263 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" Jul 21 09:01:30.745294 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.745268 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv" event={"ID":"0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93","Type":"ContainerDied","Data":"02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624"} Jul 21 09:01:30.745824 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.745312 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02e5cc52c2907c5bb3d43d37656d4aad9f3f819436096b98aee90c4074fac624" Jul 21 09:01:30.747309 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:30.747284 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerStarted","Data":"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9"} Jul 21 09:01:31.310380 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:31.310338 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:31.310650 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:31.310625 2569 patch_prober.go:28] interesting pod/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 21 09:01:31.310728 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:31.310710 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 21 09:01:32.310149 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:32.310108 2569 patch_prober.go:28] interesting pod/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 21 09:01:32.310576 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:32.310173 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 21 09:01:33.310377 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:33.310333 2569 patch_prober.go:28] interesting pod/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 21 09:01:33.310894 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:33.310414 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 21 09:01:34.310670 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:34.310631 2569 patch_prober.go:28] interesting pod/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 21 09:01:34.311101 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:34.310723 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 21 09:01:35.313375 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:35.313295 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:35.313814 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:35.313671 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:35.314380 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:35.314364 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:01:46.445709 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:46.445647 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv"] Jul 21 09:01:46.448262 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:46.448236 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b88480-m9bfv"] Jul 21 09:01:47.835913 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:47.835881 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" path="/var/lib/kubelet/pods/0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93/volumes" Jul 21 09:01:55.660437 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.660392 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2"] Jul 21 09:01:55.660878 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.660794 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" containerName="revoke-keys" Jul 21 09:01:55.660878 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.660807 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" containerName="revoke-keys" Jul 21 09:01:55.660961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.660889 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d1e4cc1-3c79-40cd-b5b3-29e7ec64ce93" containerName="revoke-keys" Jul 21 09:01:55.663992 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.663976 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:55.666770 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.666746 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:01:55.667530 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.667516 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:01:55.685786 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.685755 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2"] Jul 21 09:01:55.775285 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.775244 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2\" (UID: \"42202243-54e0-49e8-839f-a9aba672ddd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:55.876597 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.876562 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2\" (UID: \"42202243-54e0-49e8-839f-a9aba672ddd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:55.877316 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.877289 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2\" (UID: \"42202243-54e0-49e8-839f-a9aba672ddd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:55.974740 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:55.974633 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:56.334741 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:56.334716 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2"] Jul 21 09:01:56.336422 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:01:56.336391 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42202243_54e0_49e8_839f_a9aba672ddd7.slice/crio-d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7 WatchSource:0}: Error finding container d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7: Status 404 returned error can't find the container with id d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7 Jul 21 09:01:56.885759 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:56.885720 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" event={"ID":"42202243-54e0-49e8-839f-a9aba672ddd7","Type":"ContainerStarted","Data":"0cbb0afbc6d381ab09bfc6e52910d9334b00d82ae2546b63a835b663fc21b366"} Jul 21 09:01:56.886168 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:56.885775 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" event={"ID":"42202243-54e0-49e8-839f-a9aba672ddd7","Type":"ContainerStarted","Data":"d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7"} Jul 21 09:01:57.893222 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:57.893189 2569 generic.go:356] "Generic (PLEG): container finished" podID="42202243-54e0-49e8-839f-a9aba672ddd7" containerID="0cbb0afbc6d381ab09bfc6e52910d9334b00d82ae2546b63a835b663fc21b366" exitCode=0 Jul 21 09:01:57.893754 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:57.893248 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" event={"ID":"42202243-54e0-49e8-839f-a9aba672ddd7","Type":"ContainerDied","Data":"0cbb0afbc6d381ab09bfc6e52910d9334b00d82ae2546b63a835b663fc21b366"} Jul 21 09:01:59.036064 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.036040 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:59.103305 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.103272 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca\") pod \"42202243-54e0-49e8-839f-a9aba672ddd7\" (UID: \"42202243-54e0-49e8-839f-a9aba672ddd7\") " Jul 21 09:01:59.103632 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.103608 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42202243-54e0-49e8-839f-a9aba672ddd7" (UID: "42202243-54e0-49e8-839f-a9aba672ddd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:01:59.204786 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.204699 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42202243-54e0-49e8-839f-a9aba672ddd7-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:01:59.904902 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.904871 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" Jul 21 09:01:59.904902 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.904896 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2" event={"ID":"42202243-54e0-49e8-839f-a9aba672ddd7","Type":"ContainerDied","Data":"d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7"} Jul 21 09:01:59.905125 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:01:59.904944 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1764a76cd64e300212abd31bf81b3908f6316e6bb19c737821902c670133ae7" Jul 21 09:02:15.835816 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:15.835776 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2"] Jul 21 09:02:15.835816 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:15.835818 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bb8bc8-w2vb2"] Jul 21 09:02:17.834466 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:17.834416 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42202243-54e0-49e8-839f-a9aba672ddd7" path="/var/lib/kubelet/pods/42202243-54e0-49e8-839f-a9aba672ddd7/volumes" Jul 21 09:02:22.609656 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:22.609619 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:02:22.610218 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:22.609876 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" containerID="cri-o://9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9" gracePeriod=30 Jul 21 09:02:27.864735 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.864704 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:02:27.976805 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976710 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.976805 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976773 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976812 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976841 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976859 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976908 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv42n\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976934 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.976976 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977048 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977035 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo\") pod \"4d0e52bc-7618-4593-b135-fe69ca5fb916\" (UID: \"4d0e52bc-7618-4593-b135-fe69ca5fb916\") " Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977128 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977200 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977264 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977370 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-certs\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977389 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-workload-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:27.977414 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977404 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-credential-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:27.977940 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977535 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:02:27.977940 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.977829 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:02:27.979440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.979413 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 09:02:27.979566 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.979444 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n" (OuterVolumeSpecName: "kube-api-access-gv42n") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "kube-api-access-gv42n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:02:27.979820 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.979801 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:02:27.979906 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:27.979871 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d0e52bc-7618-4593-b135-fe69ca5fb916" (UID: "4d0e52bc-7618-4593-b135-fe69ca5fb916"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:02:28.055905 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.055865 2569 generic.go:356] "Generic (PLEG): container finished" podID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerID="9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9" exitCode=0 Jul 21 09:02:28.056074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.055920 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" Jul 21 09:02:28.056074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.055918 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerDied","Data":"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9"} Jul 21 09:02:28.056074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.055986 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6" event={"ID":"4d0e52bc-7618-4593-b135-fe69ca5fb916","Type":"ContainerDied","Data":"8cb83f788701316f910cc3a779c9f05cbea67f1553e6092d412fef3a980df459"} Jul 21 09:02:28.056074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.056011 2569 scope.go:117] "RemoveContainer" containerID="9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9" Jul 21 09:02:28.065733 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.065674 2569 scope.go:117] "RemoveContainer" containerID="a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55" Jul 21 09:02:28.074159 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.074141 2569 scope.go:117] "RemoveContainer" containerID="9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9" Jul 21 09:02:28.074418 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:02:28.074398 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9\": container with ID starting with 9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9 not found: ID does not exist" containerID="9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9" Jul 21 09:02:28.074485 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.074424 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9"} err="failed to get container status \"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9\": rpc error: code = NotFound desc = could not find container \"9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9\": container with ID starting with 9ac45bce3eb5348fcedb458ddd853148c5b497f7f31b0d8410e8c80e1100b6e9 not found: ID does not exist" Jul 21 09:02:28.074485 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.074441 2569 scope.go:117] "RemoveContainer" containerID="a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55" Jul 21 09:02:28.074665 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:02:28.074646 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55\": container with ID starting with a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55 not found: ID does not exist" containerID="a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55" Jul 21 09:02:28.074728 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.074668 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55"} err="failed to get container status \"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55\": rpc error: code = NotFound desc = could not find container \"a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55\": container with ID starting with a08d58ba973f623c3fe409b4b2e04ed03bb2c19d8c8d3babf2263cc117456e55 not found: ID does not exist" Jul 21 09:02:28.078139 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078118 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-envoy\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.078139 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078141 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d0e52bc-7618-4593-b135-fe69ca5fb916-istiod-ca-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.078282 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078151 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv42n\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-kube-api-access-gv42n\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.078282 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078160 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-token\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.078282 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078168 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-podinfo\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.078282 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.078175 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d0e52bc-7618-4593-b135-fe69ca5fb916-istio-data\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:02:28.086757 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.086736 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:02:28.095375 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:28.095344 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-bb8bc8-openshift-default-7d9c84946c-8b6m6"] Jul 21 09:02:29.834773 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:29.834737 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" path="/var/lib/kubelet/pods/4d0e52bc-7618-4593-b135-fe69ca5fb916/volumes" Jul 21 09:02:32.304439 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.304401 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:02:32.304961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.304946 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305006 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.304965 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305006 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.304981 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305006 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.304990 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305098 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.305013 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42202243-54e0-49e8-839f-a9aba672ddd7" containerName="revoke-keys" Jul 21 09:02:32.305098 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.305033 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="42202243-54e0-49e8-839f-a9aba672ddd7" containerName="revoke-keys" Jul 21 09:02:32.305163 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.305126 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305163 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.305136 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0e52bc-7618-4593-b135-fe69ca5fb916" containerName="istio-proxy" Jul 21 09:02:32.305163 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.305146 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="42202243-54e0-49e8-839f-a9aba672ddd7" containerName="revoke-keys" Jul 21 09:02:32.310099 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.310077 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.312530 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.312510 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-26961c-openshift-default-dockercfg-s864b\"" Jul 21 09:02:32.313416 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313393 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313425 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhnr2\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313445 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313506 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313541 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313573 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313592 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313591 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313949 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313731 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.313949 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.313780 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.320346 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.320325 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:02:32.415182 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415151 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415182 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415192 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415254 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415293 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415324 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415358 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhnr2\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415386 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415440 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415430 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415817 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415464 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415817 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415623 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.415817 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415705 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.416012 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415890 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.416012 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415953 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.416012 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.415995 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.418280 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.418244 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.418394 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.418297 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.425516 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.425481 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.425856 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.425836 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhnr2\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2\") pod \"e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.622421 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.622329 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:32.823004 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.822974 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:02:32.824304 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:02:32.824279 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf979acf2_5a87_48f8_8646_d9a51bae0883.slice/crio-9149d1e71135702ecd49ed6bfe4cf9037641c1abcb124cc9817d92bf53cf5a8d WatchSource:0}: Error finding container 9149d1e71135702ecd49ed6bfe4cf9037641c1abcb124cc9817d92bf53cf5a8d: Status 404 returned error can't find the container with id 9149d1e71135702ecd49ed6bfe4cf9037641c1abcb124cc9817d92bf53cf5a8d Jul 21 09:02:32.826639 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.826600 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:32.826751 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.826715 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:32.826799 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:32.826764 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:33.084902 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:33.084844 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerStarted","Data":"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43"} Jul 21 09:02:33.084902 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:33.084910 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerStarted","Data":"9149d1e71135702ecd49ed6bfe4cf9037641c1abcb124cc9817d92bf53cf5a8d"} Jul 21 09:02:33.167790 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:33.167674 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podStartSLOduration=1.167651076 podStartE2EDuration="1.167651076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:02:33.166154962 +0000 UTC m=+2615.829424186" watchObservedRunningTime="2026-07-21 09:02:33.167651076 +0000 UTC m=+2615.830920300" Jul 21 09:02:33.623006 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:33.622971 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:33.627914 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:33.627888 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:34.089538 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:34.089505 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:34.090498 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:34.090477 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:50.077941 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.077906 2569 patch_prober.go:28] interesting pod/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:42844->10.133.0.52:15021: read: connection reset by peer" start-of-body= Jul 21 09:02:50.078328 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.077979 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:42844->10.133.0.52:15021: read: connection reset by peer" Jul 21 09:02:50.180026 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.179992 2569 generic.go:356] "Generic (PLEG): container finished" podID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerID="d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43" exitCode=0 Jul 21 09:02:50.180233 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.180058 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerDied","Data":"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43"} Jul 21 09:02:50.180506 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.180488 2569 scope.go:117] "RemoveContainer" containerID="d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43" Jul 21 09:02:50.181627 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.181592 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:50.181739 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.181702 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:50.185697 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:50.182195 2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 21 09:02:51.191885 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:51.191817 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerStarted","Data":"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9"} Jul 21 09:02:51.623325 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:51.623279 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:51.623554 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:51.623524 2569 patch_prober.go:28] interesting pod/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 21 09:02:51.623638 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:51.623587 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 21 09:02:52.623049 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:52.623011 2569 patch_prober.go:28] interesting pod/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 21 09:02:52.623466 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:52.623090 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 21 09:02:53.623481 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:53.623441 2569 patch_prober.go:28] interesting pod/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 21 09:02:53.623945 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:53.623501 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 21 09:02:54.623226 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:54.623186 2569 patch_prober.go:28] interesting pod/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 21 09:02:54.623410 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:54.623246 2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 21 09:02:55.627074 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:55.627042 2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:55.627449 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:55.627315 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:55.627944 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:55.627929 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:02:58.102734 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:02:58.102700 2569 scope.go:117] "RemoveContainer" containerID="8912cb6c683d1db8dcc358cadb1663990ae1d469e56357f271f6212e47855cd9" Jul 21 09:03:58.679715 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:03:58.679599 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 09:03:58.682408 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:03:58.682384 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 09:03:58.687504 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:03:58.687482 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 09:03:58.690241 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:03:58.690223 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 09:06:58.150654 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:06:58.150576 2569 scope.go:117] "RemoveContainer" containerID="1b9900f97ddd779da5cc440b93406b0bea164b2b9ba7475263699a1ed8bada05" Jul 21 09:06:58.159228 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:06:58.159209 2569 scope.go:117] "RemoveContainer" containerID="584a10f100a3e23ee2e29ae6f8f74e3fc51d8d7a90c531252e31ecb3a2ae8e18" Jul 21 09:06:58.168282 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:06:58.168263 2569 scope.go:117] "RemoveContainer" containerID="9fbcf4efa1cc0d808eab311f42f520f415e9437772b54cf6b2e9bd1995788108" Jul 21 09:06:58.175888 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:06:58.175866 2569 scope.go:117] "RemoveContainer" containerID="e5fbb54c36fceb7919cfcff7498e9c1f9ae2ca27cbbd0acf84e54b06b7ec799b" Jul 21 09:07:58.207161 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:07:58.207128 2569 scope.go:117] "RemoveContainer" containerID="0cbb0afbc6d381ab09bfc6e52910d9334b00d82ae2546b63a835b663fc21b366" Jul 21 09:07:58.215770 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:07:58.215745 2569 scope.go:117] "RemoveContainer" containerID="21ce7cfa523812b63875b9f9c1307021afe872a67ba1297627abfd629a1f0d6a" Jul 21 09:08:58.347922 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.347840 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd"] Jul 21 09:08:58.354761 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.353665 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:08:58.358070 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.358050 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:08:58.358070 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.358065 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:08:58.367636 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.367602 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd"] Jul 21 09:08:58.495773 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.495740 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-824408-bxwmd\" (UID: \"c5ff26aa-81ce-4905-a45c-539f330176a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:08:58.596451 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.596420 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-824408-bxwmd\" (UID: \"c5ff26aa-81ce-4905-a45c-539f330176a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:08:58.597111 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.597084 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-824408-bxwmd\" (UID: \"c5ff26aa-81ce-4905-a45c-539f330176a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:08:58.664296 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.664232 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:08:58.710538 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.710513 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 09:08:58.715278 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.715254 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 09:08:58.721447 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.721427 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 09:08:58.727438 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.727416 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kt5wl_4c583c41-6a5e-478d-84b6-a247cb285603/ovn-acl-logging/0.log" Jul 21 09:08:58.809654 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:58.809629 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd"] Jul 21 09:08:58.811630 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:08:58.811602 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ff26aa_81ce_4905_a45c_539f330176a0.slice/crio-643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c WatchSource:0}: Error finding container 643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c: Status 404 returned error can't find the container with id 643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c Jul 21 09:08:59.044203 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:59.044170 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" event={"ID":"c5ff26aa-81ce-4905-a45c-539f330176a0","Type":"ContainerStarted","Data":"744492af95d220f6eba0bae2aef3d5f0b74b23039bf8fa60cf8afc84b7ecaf5f"} Jul 21 09:08:59.044323 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:59.044228 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" event={"ID":"c5ff26aa-81ce-4905-a45c-539f330176a0","Type":"ContainerStarted","Data":"643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c"} Jul 21 09:08:59.076177 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:08:59.076131 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" podStartSLOduration=1.076115856 podStartE2EDuration="1.076115856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 09:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:08:59.075065712 +0000 UTC m=+3001.738334938" watchObservedRunningTime="2026-07-21 09:08:59.076115856 +0000 UTC m=+3001.739385080" Jul 21 09:09:00.050515 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:00.050483 2569 generic.go:356] "Generic (PLEG): container finished" podID="c5ff26aa-81ce-4905-a45c-539f330176a0" containerID="744492af95d220f6eba0bae2aef3d5f0b74b23039bf8fa60cf8afc84b7ecaf5f" exitCode=0 Jul 21 09:09:00.051027 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:00.050539 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" event={"ID":"c5ff26aa-81ce-4905-a45c-539f330176a0","Type":"ContainerDied","Data":"744492af95d220f6eba0bae2aef3d5f0b74b23039bf8fa60cf8afc84b7ecaf5f"} Jul 21 09:09:01.193280 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:01.193255 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:09:01.316747 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:01.316629 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca\") pod \"c5ff26aa-81ce-4905-a45c-539f330176a0\" (UID: \"c5ff26aa-81ce-4905-a45c-539f330176a0\") " Jul 21 09:09:01.316985 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:01.316961 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5ff26aa-81ce-4905-a45c-539f330176a0" (UID: "c5ff26aa-81ce-4905-a45c-539f330176a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:09:01.417934 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:01.417894 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ff26aa-81ce-4905-a45c-539f330176a0-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:02.064331 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:02.064301 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" event={"ID":"c5ff26aa-81ce-4905-a45c-539f330176a0","Type":"ContainerDied","Data":"643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c"} Jul 21 09:09:02.064331 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:02.064325 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd" Jul 21 09:09:02.064539 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:02.064348 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="643ffb606ba8868c5cd2546fb727026b085ecf348cc1cb18831299fc89d5101c" Jul 21 09:09:13.451470 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:13.451432 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd"] Jul 21 09:09:13.458586 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:13.458560 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-824408-bxwmd"] Jul 21 09:09:13.834024 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:13.833991 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ff26aa-81ce-4905-a45c-539f330176a0" path="/var/lib/kubelet/pods/c5ff26aa-81ce-4905-a45c-539f330176a0/volumes" Jul 21 09:09:20.935235 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.935203 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88"] Jul 21 09:09:20.935715 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.935574 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ff26aa-81ce-4905-a45c-539f330176a0" containerName="revoke-keys" Jul 21 09:09:20.935715 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.935585 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ff26aa-81ce-4905-a45c-539f330176a0" containerName="revoke-keys" Jul 21 09:09:20.935715 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.935665 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ff26aa-81ce-4905-a45c-539f330176a0" containerName="revoke-keys" Jul 21 09:09:20.938632 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.938616 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:20.941194 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.941174 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zqr2c\"" Jul 21 09:09:20.941443 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.941427 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 09:09:20.946710 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.946674 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88"] Jul 21 09:09:20.979932 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:20.979905 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26961c-lrm88\" (UID: \"8a5aea18-e425-4682-913f-884ee69156bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:21.080418 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:21.080383 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26961c-lrm88\" (UID: \"8a5aea18-e425-4682-913f-884ee69156bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:21.080973 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:21.080956 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26961c-lrm88\" (UID: \"8a5aea18-e425-4682-913f-884ee69156bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:21.250837 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:21.250816 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:21.385455 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:21.385427 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88"] Jul 21 09:09:21.387838 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:09:21.387808 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5aea18_e425_4682_913f_884ee69156bd.slice/crio-c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae WatchSource:0}: Error finding container c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae: Status 404 returned error can't find the container with id c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae Jul 21 09:09:22.162452 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:22.162417 2569 generic.go:356] "Generic (PLEG): container finished" podID="8a5aea18-e425-4682-913f-884ee69156bd" containerID="cfff203b7d618643c771769ea1a34da5747ac2970463ea93f6062766fc64b550" exitCode=0 Jul 21 09:09:22.162807 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:22.162498 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" event={"ID":"8a5aea18-e425-4682-913f-884ee69156bd","Type":"ContainerDied","Data":"cfff203b7d618643c771769ea1a34da5747ac2970463ea93f6062766fc64b550"} Jul 21 09:09:22.162807 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:22.162551 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" event={"ID":"8a5aea18-e425-4682-913f-884ee69156bd","Type":"ContainerStarted","Data":"c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae"} Jul 21 09:09:23.300896 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:23.300875 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:23.400582 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:23.400556 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca\") pod \"8a5aea18-e425-4682-913f-884ee69156bd\" (UID: \"8a5aea18-e425-4682-913f-884ee69156bd\") " Jul 21 09:09:23.400960 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:23.400937 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8a5aea18-e425-4682-913f-884ee69156bd" (UID: "8a5aea18-e425-4682-913f-884ee69156bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:09:23.501620 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:23.501598 2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a5aea18-e425-4682-913f-884ee69156bd-maas-api-service-ca\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:24.175949 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:24.175859 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" event={"ID":"8a5aea18-e425-4682-913f-884ee69156bd","Type":"ContainerDied","Data":"c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae"} Jul 21 09:09:24.175949 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:24.175880 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88" Jul 21 09:09:24.175949 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:24.175913 2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6ae10d689972d7c9671aecf8fdfd80c0df59f52a60501f42de3a69ccb7ed8ae" Jul 21 09:09:36.036778 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:36.036735 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88"] Jul 21 09:09:36.038288 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:36.038262 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26961c-lrm88"] Jul 21 09:09:37.833882 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:37.833849 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a5aea18-e425-4682-913f-884ee69156bd" path="/var/lib/kubelet/pods/8a5aea18-e425-4682-913f-884ee69156bd/volumes" Jul 21 09:09:43.006235 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:43.006200 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:09:43.006753 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:43.006489 2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" containerID="cri-o://d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9" gracePeriod=30 Jul 21 09:09:48.257477 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.257450 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:09:48.290022 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.289957 2569 generic.go:356] "Generic (PLEG): container finished" podID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerID="d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9" exitCode=0 Jul 21 09:09:48.290022 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.290010 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerDied","Data":"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9"} Jul 21 09:09:48.290022 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.290015 2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" Jul 21 09:09:48.290235 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.290083 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw" event={"ID":"f979acf2-5a87-48f8-8646-d9a51bae0883","Type":"ContainerDied","Data":"9149d1e71135702ecd49ed6bfe4cf9037641c1abcb124cc9817d92bf53cf5a8d"} Jul 21 09:09:48.290235 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.290107 2569 scope.go:117] "RemoveContainer" containerID="d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9" Jul 21 09:09:48.298712 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.298693 2569 scope.go:117] "RemoveContainer" containerID="d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43" Jul 21 09:09:48.306784 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.306764 2569 scope.go:117] "RemoveContainer" containerID="d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9" Jul 21 09:09:48.307052 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:09:48.307023 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9\": container with ID starting with d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9 not found: ID does not exist" containerID="d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9" Jul 21 09:09:48.307098 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.307062 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9"} err="failed to get container status \"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9\": rpc error: code = NotFound desc = could not find container \"d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9\": container with ID starting with d8d3eb0738fdfd12b466d466226e68ed59d7a687b3eea87c056d3224f13456d9 not found: ID does not exist" Jul 21 09:09:48.307098 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.307081 2569 scope.go:117] "RemoveContainer" containerID="d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43" Jul 21 09:09:48.307273 ip-10-0-142-97 kubenswrapper[2569]: E0721 09:09:48.307257 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43\": container with ID starting with d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43 not found: ID does not exist" containerID="d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43" Jul 21 09:09:48.307318 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.307277 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43"} err="failed to get container status \"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43\": rpc error: code = NotFound desc = could not find container \"d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43\": container with ID starting with d5321aa70882e32a0b2b4ced145ad053a36ed3303daf91bdf632946eb2382b43 not found: ID does not exist" Jul 21 09:09:48.431472 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431433 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431472 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431475 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431544 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431571 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431646 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431675 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhnr2\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431721 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.431785 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431774 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.432025 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431811 2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert\") pod \"f979acf2-5a87-48f8-8646-d9a51bae0883\" (UID: \"f979acf2-5a87-48f8-8646-d9a51bae0883\") " Jul 21 09:09:48.432025 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431828 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:09:48.432025 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.431848 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:09:48.432182 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.432072 2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-certs\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.432182 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.432090 2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-credential-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.432283 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.432263 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:09:48.432283 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.432269 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 09:09:48.432563 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.432461 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data" (OuterVolumeSpecName: "istio-data") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:09:48.434043 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.433993 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2" (OuterVolumeSpecName: "kube-api-access-lhnr2") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "kube-api-access-lhnr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:09:48.434157 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.434055 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 09:09:48.434355 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.434334 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 09:09:48.434416 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.434400 2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token" (OuterVolumeSpecName: "istio-token") pod "f979acf2-5a87-48f8-8646-d9a51bae0883" (UID: "f979acf2-5a87-48f8-8646-d9a51bae0883"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 09:09:48.533305 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533264 2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-workload-socket\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533305 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533301 2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-podinfo\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533305 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533311 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhnr2\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-kube-api-access-lhnr2\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533305 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533319 2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-envoy\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533556 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533328 2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-token\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533556 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533334 2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f979acf2-5a87-48f8-8646-d9a51bae0883-istiod-ca-cert\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.533556 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.533342 2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f979acf2-5a87-48f8-8646-d9a51bae0883-istio-data\") on node \"ip-10-0-142-97.ec2.internal\" DevicePath \"\"" Jul 21 09:09:48.619661 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.619638 2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:09:48.621624 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:48.621603 2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-26961c-openshift-default-6f89699987-h4cnw"] Jul 21 09:09:49.835150 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:49.835118 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" path="/var/lib/kubelet/pods/f979acf2-5a87-48f8-8646-d9a51bae0883/volumes" Jul 21 09:09:50.430436 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:50.430407 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787895fdf5-s8g66_20479057-0345-42d2-934d-96620bb4ffe1/copy-manifests/0.log" Jul 21 09:09:50.438057 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:50.438038 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787895fdf5-s8g66_20479057-0345-42d2-934d-96620bb4ffe1/manager/0.log" Jul 21 09:09:50.930357 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:50.930326 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743725-fsb76_fd30d57f-75b4-472b-a1c4-3e9c649e45a9/cleanup/0.log" Jul 21 09:09:51.056230 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:51.056202 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743740-4b9mx_924a9391-6029-4055-a246-f4e02cf7a48a/cleanup/0.log" Jul 21 09:09:53.099351 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:53.099319 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jxb5w_7c93798f-65a1-42e4-be56-d1d776a176ef/manager/0.log" Jul 21 09:09:53.797368 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:53.797333 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bb4pw_188c5f34-1541-47a6-84ae-bacc24668576/discovery/0.log" Jul 21 09:09:55.080836 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:55.080804 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hc6wb_0b6eedde-c4e6-4268-a753-1c796862c919/main/0.log" Jul 21 09:09:55.338805 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:09:55.338701 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6zqfj_ee4c7b15-c6b0-4425-9d86-8fb206742962/main/0.log" Jul 21 09:10:04.457232 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:04.457197 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jkrrd_de0d6800-e8cd-4757-b2b7-9ee26630ec2a/global-pull-secret-syncer/0.log" Jul 21 09:10:04.582862 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:04.582832 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r7kh5_5291dae6-6ea3-4477-92a1-25b390c568e3/konnectivity-agent/0.log" Jul 21 09:10:04.660341 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:04.660312 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-97.ec2.internal_3259959531b2cae4a95970f39ca273a7/haproxy/0.log" Jul 21 09:10:10.369198 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:10.369165 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jxb5w_7c93798f-65a1-42e4-be56-d1d776a176ef/manager/0.log" Jul 21 09:10:12.176794 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.176769 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/alertmanager/0.log" Jul 21 09:10:12.204790 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.204766 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/config-reloader/0.log" Jul 21 09:10:12.228247 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.228222 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/kube-rbac-proxy-web/0.log" Jul 21 09:10:12.253828 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.253802 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/kube-rbac-proxy/0.log" Jul 21 09:10:12.281140 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.281119 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/kube-rbac-proxy-metric/0.log" Jul 21 09:10:12.308457 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.308435 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/prom-label-proxy/0.log" Jul 21 09:10:12.332719 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.332694 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7ad07d97-7a48-48e8-9974-493b640cb220/init-config-reloader/0.log" Jul 21 09:10:12.397220 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.397197 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-n62bp_b9866716-ac4a-4838-b924-46cddc5d44f0/cluster-monitoring-operator/0.log" Jul 21 09:10:12.443495 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.443423 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mjtpp_bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43/kube-state-metrics/0.log" Jul 21 09:10:12.490324 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.490304 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mjtpp_bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43/kube-rbac-proxy-main/0.log" Jul 21 09:10:12.538725 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.538705 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-mjtpp_bbcbc716-3fcc-40c0-acda-2d3dd6f3bb43/kube-rbac-proxy-self/0.log" Jul 21 09:10:12.842753 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.842730 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz96j_e206cc46-0d54-4324-b528-85142c32ee87/node-exporter/0.log" Jul 21 09:10:12.914545 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.914512 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz96j_e206cc46-0d54-4324-b528-85142c32ee87/kube-rbac-proxy/0.log" Jul 21 09:10:12.973734 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:12.973706 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz96j_e206cc46-0d54-4324-b528-85142c32ee87/init-textfile/0.log" Jul 21 09:10:13.214911 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.214838 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zq7tl_eda7ae38-e803-48a8-a915-8f09489ad530/kube-rbac-proxy-main/0.log" Jul 21 09:10:13.261752 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.261719 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zq7tl_eda7ae38-e803-48a8-a915-8f09489ad530/kube-rbac-proxy-self/0.log" Jul 21 09:10:13.312446 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.312423 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-zq7tl_eda7ae38-e803-48a8-a915-8f09489ad530/openshift-state-metrics/0.log" Jul 21 09:10:13.806232 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.806209 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cb5854f7-88c5z_956e0144-e6cd-4acd-8d40-d95fd63fcdae/telemeter-client/0.log" Jul 21 09:10:13.839798 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.839771 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cb5854f7-88c5z_956e0144-e6cd-4acd-8d40-d95fd63fcdae/reload/0.log" Jul 21 09:10:13.874078 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:13.874054 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cb5854f7-88c5z_956e0144-e6cd-4acd-8d40-d95fd63fcdae/kube-rbac-proxy/0.log" Jul 21 09:10:14.477529 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477494 2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz"] Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477874 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a5aea18-e425-4682-913f-884ee69156bd" containerName="revoke-keys" Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477886 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a5aea18-e425-4682-913f-884ee69156bd" containerName="revoke-keys" Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477903 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477908 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477923 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.477961 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.477930 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.478415 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.478391 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a5aea18-e425-4682-913f-884ee69156bd" containerName="revoke-keys" Jul 21 09:10:14.478492 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.478461 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.478492 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.478473 2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f979acf2-5a87-48f8-8646-d9a51bae0883" containerName="istio-proxy" Jul 21 09:10:14.483035 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.483014 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.486562 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.486538 2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xts55\"/\"default-dockercfg-zhkmm\"" Jul 21 09:10:14.487720 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.487699 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xts55\"/\"kube-root-ca.crt\"" Jul 21 09:10:14.488225 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.488211 2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xts55\"/\"openshift-service-ca.crt\"" Jul 21 09:10:14.495268 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.495246 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz"] Jul 21 09:10:14.548217 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.548184 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-lib-modules\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.548364 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.548240 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-proc\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.548364 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.548290 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-podres\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.548364 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.548334 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-sys\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.548364 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.548353 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8897\" (UniqueName: \"kubernetes.io/projected/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-kube-api-access-h8897\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.648875 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.648837 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-sys\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649057 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.648888 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8897\" (UniqueName: \"kubernetes.io/projected/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-kube-api-access-h8897\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649057 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.648954 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-lib-modules\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649057 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.648958 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-sys\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649057 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.649044 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-proc\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649259 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.649070 2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-podres\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649259 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.649103 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-lib-modules\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649259 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.649104 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-proc\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.649259 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.649182 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-podres\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.659287 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.659256 2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8897\" (UniqueName: \"kubernetes.io/projected/cfa3c6c7-6286-4e49-b7a2-327d59c6914e-kube-api-access-h8897\") pod \"perf-node-gather-daemonset-zbkgz\" (UID: \"cfa3c6c7-6286-4e49-b7a2-327d59c6914e\") " pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.794481 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.794449 2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:14.928179 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:14.928142 2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz"] Jul 21 09:10:14.932751 ip-10-0-142-97 kubenswrapper[2569]: W0721 09:10:14.932720 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcfa3c6c7_6286_4e49_b7a2_327d59c6914e.slice/crio-2d8c4bcceb850ad283e4eb768452cb96a48448cfa1e0b2fe9f6750fdb89888a8 WatchSource:0}: Error finding container 2d8c4bcceb850ad283e4eb768452cb96a48448cfa1e0b2fe9f6750fdb89888a8: Status 404 returned error can't find the container with id 2d8c4bcceb850ad283e4eb768452cb96a48448cfa1e0b2fe9f6750fdb89888a8 Jul 21 09:10:15.419618 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:15.419537 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" event={"ID":"cfa3c6c7-6286-4e49-b7a2-327d59c6914e","Type":"ContainerStarted","Data":"f0b376d9712cdc1d58ac2bbd619658e4ddffe977fd14568efb890c780a457134"} Jul 21 09:10:15.419618 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:15.419585 2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" event={"ID":"cfa3c6c7-6286-4e49-b7a2-327d59c6914e","Type":"ContainerStarted","Data":"2d8c4bcceb850ad283e4eb768452cb96a48448cfa1e0b2fe9f6750fdb89888a8"} Jul 21 09:10:15.419834 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:15.419702 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:15.925652 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:15.925624 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/2.log" Jul 21 09:10:15.930052 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:15.930032 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2k5vl_bc47e7ac-e08a-464e-be52-f8791476c928/console-operator/3.log" Jul 21 09:10:16.441930 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:16.441898 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-c7vr5_397f1417-0b0a-4d74-94c1-704f3b33eee5/download-server/0.log" Jul 21 09:10:17.060874 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:17.060845 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-sjz2l_5542bb62-ebd1-458a-bfb8-f8d1df5bb431/volume-data-source-validator/0.log" Jul 21 09:10:18.228742 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:18.228712 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7qwx_19415ce2-6c93-40de-9e76-02f21f6dd80b/dns/0.log" Jul 21 09:10:18.251012 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:18.250989 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7qwx_19415ce2-6c93-40de-9e76-02f21f6dd80b/kube-rbac-proxy/0.log" Jul 21 09:10:18.370315 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:18.370286 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpg6q_4e9d46a3-87c8-4b06-a13f-3fb6e3f219a4/dns-node-resolver/0.log" Jul 21 09:10:19.007400 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:19.007374 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b68679575-75d6q_6e81384f-2d5d-4329-9194-49a40988c4b4/registry/0.log" Jul 21 09:10:19.087051 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:19.087021 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tmmt9_3e35a47c-9f4f-4988-abe0-c7575f829bea/node-ca/0.log" Jul 21 09:10:20.240930 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:20.240893 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bb4pw_188c5f34-1541-47a6-84ae-bacc24668576/discovery/0.log" Jul 21 09:10:21.262710 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.262658 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-htv2f_b2f25635-fec1-46ac-9678-4ba245329c84/serve-healthcheck-canary/0.log" Jul 21 09:10:21.434435 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.434407 2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" Jul 21 09:10:21.454939 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.454894 2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xts55/perf-node-gather-daemonset-zbkgz" podStartSLOduration=7.454874751 podStartE2EDuration="7.454874751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 09:10:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 09:10:15.438105627 +0000 UTC m=+3078.101374844" watchObservedRunningTime="2026-07-21 09:10:21.454874751 +0000 UTC m=+3084.118144025" Jul 21 09:10:21.823147 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.823116 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7thh8_6c147d75-3b2f-47f8-bd29-471adfa4f882/kube-rbac-proxy/0.log" Jul 21 09:10:21.847567 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.847539 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7thh8_6c147d75-3b2f-47f8-bd29-471adfa4f882/exporter/0.log" Jul 21 09:10:21.870018 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:21.869982 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7thh8_6c147d75-3b2f-47f8-bd29-471adfa4f882/extractor/0.log" Jul 21 09:10:24.824128 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:24.824097 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bb4pw_188c5f34-1541-47a6-84ae-bacc24668576/discovery/0.log" Jul 21 09:10:25.632040 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:25.632008 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787895fdf5-s8g66_20479057-0345-42d2-934d-96620bb4ffe1/manager/0.log" Jul 21 09:10:25.667519 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:25.667492 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-787895fdf5-s8g66_20479057-0345-42d2-934d-96620bb4ffe1/copy-manifests/0.log" Jul 21 09:10:26.399703 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:26.399655 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-bb4pw_188c5f34-1541-47a6-84ae-bacc24668576/discovery/0.log" Jul 21 09:10:27.010363 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:27.010336 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7686776559-hnscw_aa191af6-81b7-499f-b68a-df04da73022f/manager/0.log" Jul 21 09:10:27.114763 ip-10-0-142-97 kubenswrapper[2569]: I0721 09:10:27.114734 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9wswt_fc85a4c3-c92f-4da6-99eb-1c6dc83abf19/openshift-lws-operator/0.log"